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13  abstract 

This  final  report  documents  a  rescarcli  project  that  addresses  the  impact  ol 
instructional  technology  (I.T.)  on  the  technical  training  of  military  personnel.  The 
research  was  directed  toward  (1)  measuring  benefits  that  the  Department  of  Defense 
might  gain  through  increased  utilization  of  I.T.  and  (2)  postulating  new  research 
objectives  and  programs  that  would  make  possible  the  realization  of  these  benefits. 
Potential  savings  in  cost  and  manpower  from  I.T.  were  estimated  using  a  large  data 
base,  consisting  of  actual  military  data  on  training  costs  lor  specific  courses  with 
known  student  loading  in  graduates  per  year  (labeled  throughput  based  upon  number  of 
students  graduo.ted) .  A  simple  mathematical  model  was  used  to  estimate  potential  re¬ 
ductions  in  course  time  for  samples  drawn  from  the  data  base  of  over  4000  courses. 
Exemplary  calculations  for  over  100  formal  technical  courses  were  performed. 

The  results  indicate  that  the  DoD  can  anticipate  substantially  reduced  training 
costs  by  future  utilization  of  innovations  in  I.T.  The  most  significant  variable* 
contributing  to  cost  savings  are  instructional  strategies  and  methods.  Courseware  and 
equipment  costs  were  determined  to  have  less  impact  on  cost  savings  than  was  pre¬ 
viously  considered.  The  research  results  suggest  that  the  Dob  should  expand  its  use 
of  I.T.  to  both  nontechnical  and  on-the-job  training. 

Research  in  five  areas  is  recommended:  (1)  instructional  strategies,  (2)  curric¬ 
ula  and  courseware  development,  (3)  course  evaluation  methods,  (4)  equipment  and  (5) 
benefit  and  cost  analyses.  These  exploratory  research  findings  indicate  that  a  basic 
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II. 


ABSTRACT 


This  final  report  documents  a  research  project  that  addresses  the 
impact  of  instructional  lechnologi  (I.T.)  on  the  technical  training  of 
military  personnel.  The  research  was  directed  toward  (1)  measuring 
benefits  that  t  lie  Department  of  Defense  might  gain  through  increased 
utilization  of  I.T.  and  (2)  postulating  new  research  objectives  and 
programs  that  would  make  possible  the  realization  of  these  benefits. 
Potential  savings  in  cost  and  manpower  from  applied  I.T.  were  estimated 
using  a  large  data  base,  consisting  of  actual  military  data  on  cost  and 
school  throughput,  A  simple  mathematical  model  was  used  to  estimate 
potential  reductions  in  course  time  for  samples  drawn  from  the  data  base 
of  over  4000  courses.  Exemplary  calculations  for  over  100  formal  tech¬ 
nical  courses  were  performed. 

The  results  indicate  that  t  lie  Dol)  can  anticipate  subst  ant  inllv 
reduced  training  costs  b\  future  utilization  of  innovations  in  I.T.  The 
most  significant  variables  contributing  to  cost  savings  are  instructional 
strategics  and  methods.  Courseware  and  equipment  costs  were  determined 
to  have  less  impact  on  cost  savings  than  previously  considered.  The 
research  results  suggest  that  the  DoD  should  expand  its  use  of  I.T.  to 
both  nontechnical  and  on-the-job  training. 

Research  in  five  areas  is  recommended:  (1)  instructional  strategies 
(2)  curricula  and  courseware  development,  (3)  course  evaluation  methods, 
(4)  equipment,  and  (5)  benefit  and  cost  analyses.  These  exploratory 
lescaich  findings  indicate  that  a  basic  I.T.  research  center  is  desirable 
for  the  IXiU. 
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l'REFACl 


lhls  study  wa s  Intended  to  provide  data  useful  to  tile  Defense 
Advanced  Research  Piojecls  Agency  In  formulating  its  research  program  in 
Advanced  Training  Technology. 

Tne  work  focused  on  the  estimation  of  cost  savings  resulting  from 
different  assumptions  about:  the  cost  of  delivering  training  per  student 
contract  hour;  the  cost  of  developing  training  courses  using  advanced 
training;  and  the  amount  of  reduction  in  training  time  resulting  from 
use  of  advanced  training  technology,  variably  paced  and  delivered  in  a 
form  individualized  to  meet  the  needs  of  individual  trainees. 

The  report  points  out  that  the  reader  should  not  interpret  the 
estimated  savings  literally.  The  figures  can  only  be  taken  as  indications 
of  trends  in  the  field.  The  report  qualifies  the  findings  in  a  number  of 
ways,  including; 

1.  The  course  sample  is  several  years  old  and  the  nature  of  the 
courses,  number  of  students,  and  the  course  length  have  changed 
in  a  number  of  cases  resulting  in  changes  in  potential  savings. 

2.  The  pi i„edurc  for  estimating  the  reduction  in  course  time  was 
applied  consistently  across  the  course  sample  and,  therefore, 
gives  relative  values  on  savings,  however  the  procedure  relies 
heavily  on  the  professional  .judgments  of  the  project  staff 
rather  than  direct  measurement. 

3.  The  measurement  of  course  cost  is  handled  in  different  ways 
by  the  Services  of  the  Armed  Forces. 

1.  Savings  are  dependent  on  shortening  of  course  time  which,  in 

turn,  is  dependent  on  individualization  of  instruction  resulting 
in  a  spread  over  time  of  course  completion  by  trainees.  The 
potential  savings  can  only  be  achieved  if  post  course  assignment 
procedures  can  effectively  put  students  into  productive  activity 
as  they  complete  the  course. 


xv 


The  report  does  identify  important  relationships  between  the  key 
parameters  of  reduction  in  course  time,  cost  of  delivery,  and  cost  ol 


development.  It  is  on  these  relationships  that  the  report  recommendations 
are  based. 


Thomas  C.  O'Sullivan 

Project  Director 

Advanced  Training  Technology 

Defense  Advanced  Research  Projects  Agency 
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I  INTRODUCTION 


Instructional  technology  (I.T.)  i*  beginning  to  affect  training 
within  the  '  .b.  military  establishment.  In  fact,  no  single  organization 
is  iikel\  to  push  instructional  technology  forward  on  a  broad  iron! 
taster  than  the  U.S.  military.  This  report  describes  the  findings  of  a 
research  program  conducted  by  Stanford  Research  Institute  (SRI)  ioi  the 
Advanced  Research  Frojects  Agency  (ARPA)  of  i he  Department  of  Defense 
(Don) .  The  year- long  study  focused  on  (1)  the  role  of  I. I.  in  the  U.S. 
military  training  programs  and  (2)  the  necessary  research  programs  te- 
quired  for  the  DoD  to  benefit  from  I.T. 

The  analysis  of  tiiis  role  required  the  cooperation  oi  t  lie  three 
major  military  services.  SRI  wishes  to  thank  the  various  offices  within 
the  services  for  their  invaluable  cooperation  and  contributions  to  this 
study . 

This  report  is  divided  into  six  main  set t  ions  and  several  appendices, 
Following  this  introduction  (Section  I),  Section  II  summarizes  the  tech¬ 
nical  problem,  methodology  used,  research  results,  and  implications  oi 
these  results.  Section  III  identifies  I.T.  research  programs  and  rele¬ 
vant  data  within  the  DoD.  Section  IV  presents  numerical  estimates  of 
potential  savings  from  I.T.  using  a  computational  model  based  upon  1971 
and  1972  cost  and  manpower  data.  Section  V  discusses  inferences  from 
the  computational  model,  implied  trends,  and  deductions  about  future 
research.  Section  VI  suggests  future  research  to  enable  the  DoD  to 
realize  benefits  from  I.T.  One  of  the  important  research  alternatives 
is  the  implementation  of  I.T.  research  centers.  The  appendices  cover  a 
survey  of  reported  I.T.  applications,  source  documentation  on  training 
costs  and  number  of  trainees,  a  summary  of  the  cost  calculations,  and 
representative  experimental  values  of  course  time  reductions  used  as 
calibration  data  for  the  calculations. 
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II  SUMMARY 


A.  Technical  Problem 

This  study  focuses  on  the  relationship  of  instructional  technology 
(I.T.)  to  military  training  and  the  research  that  must  be  conducted  to 
enable  the  military  to  realize  the  benefits  inherent  in  this  new  tech¬ 
nology. 

I.T.  lias  been  defined  as  "a  systematic  way  of  designing,  carrying 
out,  and  evaluating  the  total  process  of  learning  and  teaching  in  terms 
of  specific  objectives,  based  on  research  in  human  learning  and  communi¬ 
cation,  and  employing  a  combination  of  human  and  noiihuman  resources  to 

-.1* 

bring  about  more  effective  instruction.  I.T.  thus  involves  the 

combined  use  of  modern  technological  devices  and  individual  and  group 
motivating  techniques  to  improve  the  instructional  process.  These  tools 
and  techniques  include  individualized  learning,  peer  instruction,  group 
dynamics,  audio-visual  aids,  programmed  instruction,  teaching  machines, 
and  the  computer. 

Experiments  in  both  military  and  civilian  educational  institutions 
have  suggested  that  I.T.  offers  both  tangible  cost-effective  benefits 
and  others  less  tangible.  The  measurement  of  these  benefits  is  the  fi*st 
step  toward  proper  identification  of  new  channels  for  increased  utiliza¬ 
tion  of  I.T,  in  the  military. 

The  research  objectives  of  this  study  were  to  establish  criteria  for 
measuring  the  tangible  benefits  of  I.T.,  to  study  the  development  of 
I.T.  within  the  services,  to  assess  the  present  and  potential  commonal¬ 
ity  of  these  efforts,  and  to  make  recommendations  to  exploit  the  further 
benefits  of  I.T.  In  particular,  the  research  program  was  specifically 


*Refcrcnces  arc  listed  on  page  83. 
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intended  to  postulate  new  research  objectives  and  programs  that  would 
establish  a  foundation  for  obtaining  these  potential  benefits  from  I.T. 

B.  Methodology 

To  achieve  the  research  objectives,  efforts  were  directed  toward 
measuring  potential  cost  and  manpower  savings  from  applied  l.T.  This  was 
accompl ished  by : 

•  Establishing  a  data  base  consisting  of  training  costs  and 
throughputs  (graduates  per  year)  for  approximately  1000 
formal  tcchnictl  training  courses,  as  actually  recorded  by 
the  military. 

•  Identifying  the  meaningful  parameters  of  l.T. 

•  Developing  a  simple  mathematical  model  I  hat  determined 
potential  cost  and  manpower  savings. 

Before  the  potential  savings  in  dollars  and  manpower  could  be  esti¬ 
mated,  the  reductions  in  course  length  resulting  from  the  application  of 
I.T.  had  to  be  estimated.  Many  socio-psychological  variables  can  affect 
the  amount  of  course  time  a  given  student  needs  to  complete  any  single 
course  successfully.  Thirteen  variables  were  identified  as  key  contrib¬ 
utors  to  tho  expected  reductions  in  course  time.  Their  individual  effects 
were  integrated  into  a  single  parameter,  the  fractional  reduction  in 
course  time,  c(CT).  Calculations  were  performed,  based  upon  estimated 
values  of  6(CT),  that  would  likely  be  realized  if  I.T.  were  introduced 
into  course  curricula. 

A  sample  comprised  of  about  120  courses  from  all  three  services  was 
selected  lor  tho  calculations.  The  courses  were  chop  n  to  represent  a 
diversity  of  curricula,  subject  to  the  constraint  that  complete  data  on 
each  course  was  available.  Three  parameters  were  varied:  (1)  5(CT), 

(2)  I.T.  equipment  costs,  and  (3)  courseware  development  costs.  The 
estimated  cost  reductions  (ii  any)  were  quantified. 

4 


r 


Cost  and  manpower  reductions  wore  clio.sen  ns  measures  l'ov  providing 
quantitative  indications  ol  the  principal  factors  involved  in  optimizing 
training  effectiveness.  To  optimize  in  the  short  run,  t  lie  savings  poten¬ 
tial  was  based  on  a  11)71  ''snapshot"  of  the  formal  military  training 
schools  and  the  potential  for  improvement.  Since  the  implementation  of 
that  potential  will  likely  require  many  years,  it  is  expect  ml  that  new 
requ i remen t s ,  new  instructional  strategies,  and  new  technology  will  have 
to  be  considered  as  they  arise  in  the  future.  Therefore,  the  recommenda¬ 
tions  reported  herein  dose  vibe  programs  that  will  fat  ilitate  appropriate 
evaluation  of  l.T.  and  instructional  methods  in  this  eon  1  inning  evolution. 
Kstimales  of  potential  savings  are  to  lie  interpreted  as  guidelines  to  the 
itreas  of  most  effective  applications  of  new  l.T.  in  1971  to  1975  and  to 
research  opportunities.  The  estimates  should  not  be  considered  as 
predictions  ol  actual  savings  at  u  specific  t  imo. 

•  Technical  Results 

The  calculations  suggest,  that  tor  samples  of  fit)  Air  force.  39  Army, 
and  51)  Navy  courses,  annual  dollar  sav  ings  ol  $11.1  million,  $15.1)  million, 
and  $13.3  million,  respectively,  might  accrue  for  the  sampled  courses 
through  the  application  ol  l.T.  These  savings  were  calculated  based  upon 
assumed  hardware  costs  of  $t),5t)  per  terminal  hour  and  total  courseware 
development  costs  of  $2500  pet  courseware  hour  tor  each  hour  of  the 
course  adapted  to  l.T.  It  is  not  certain  that  these  projected  savings 
can  he  converted  into  actual  savings  because  questions  remain  concerning 
reassignment  practices,  options  for  increased  training,  and  other  factors. 
However,  the  estimated  savings  will  certainly  contribute  to  increased 
effectiveness  within  each  service. 

The  corresponding  mnual  manpower  savings  for  the  sampled  courses 


under  the  same  assumptions  would  be  1150,  12H0,  and  2300  man-years, 


respectively.*  Interpreting  these  figures  as  general  indicators  and  not 
as  specific  course-dependent  data,  these  results  suggest  that  implementa¬ 
tion  ol  innovations  in  1 ,T.  in  the  military  can  reduce  total  training 
costs. 

By  testing  the  sensitivity  of  the  variables  in  the  cost  model,  other 
important  results  were  observed.  Considerable  sensitivity  to  the  varia¬ 
tions  in  5(CT)  is  noted.  The  most  significant  variables  are  training 
strategics  and  instructional  methods,  which  impact  directly  on  6(CT) , 

Common  curricula  indicate  that  the  services  already  see  the  value 
of  shared  training  activities.  However,  there  are  several  advantages  to 
increased  commonality.  These  are  discussed  in  Section  V  of  the  report. 
Although  increased  commonality  is  not  necessary  to  gain  benefits  from 
I.T.,  the  project  team  concluded  that  increasing  commonality  in  applied 
I  .T .  will  result  i’  larger  cost  and  inanpow  r  savings  than  would  otherwise 
accrue . 

I).  Department  of  Defense  Implications 

In  t  lie  DoD  training  system  about  hall  ol'  the  training  occurs  on  the 
job,  and  the  rest  is  formally  organized  as  schooling.  For  all  training, 
however,  I.T.  which  uses  raining  aids  where  t  lie  trainee  is  normally 
located,  oilers  important  and  worthwhile  savings.  The  minor  redirection 
of  training  philosophy  toward  increased  utilization  ol  I.T.  that  is 
needed  to  realize  these  benefits  is  already  underway.  Examples  include 
the  basic  electricity  and  electronics  training  at  Keesler  MB,  the  Com¬ 
puterized  Training  System  (CTS)  at  Ft.  Monmouth,  and  the  new  techniques 

The  selection  ol  courses  for  the  Army  and  Air  Force  samples  was  made  on 
the  basis  of  the  available  data.  The  Navy  samples  were  biased  toward 
those  courses  where  large  manpower  savings  might  be  effected.  The 
larger  sample  size  and  the  bias  toward  manpower  savings  were  performed 
as  part  of  a  related  study  for  ONH. 


for  training  in  mathematics  and  engineering  at  the  Air  force  Academy  at 
Colorado  Springs.  A  strong  trend  exists  toward  training  in  the  field 
and  on  board  ship  or  aircraft,  and  the  idea  is  popular  among  field  com¬ 
manders.  If  the  transition  to  greater  use  of  l.T.  in  military  training 
is  made  in  a  careful,  continuous  manner,  time  and  experience  will  permit 
new  procedures,  policies,  and  equipment  to  come  into  orderly  use  without 
disrupting  normal  operations.  Thus,  SRI  recommends  that  the  DoD  continue 
to  improve  allocations  of  training  resources  by  utilizing  l.T.  throughout 
its  formal  and  informal  training  programs. 

E .  Implications  for  Future  Research 

The  potential  gains  in  dollar  and  manpower  savings  are  large  com¬ 
pared  to  the  small  cost  associated  with  the  risk  to  the  DoD  of  investing 
additional  funds  in  future  research  in  l.T.  The  potential  gains  to  the 
DoD  of  the  research  suggested  below  lie  in  its  commonality  to  all  military 
services,  in  its  fundamental  nature  (not  operational  or  developmental), 
and  in  its  ease  of  integration  with  projected  l.T.  systems.  Hie  rt'fjm- 
mchdc i  research  reflects  the  present  cost  study,  wb  results  demonstrate 
that,  of  the  many  variables  anti  parameters  studied,  instructional  strate¬ 
gies  and  methods  have  the  greatest  impact  on  cost  savings.  The  purpose  ol 
the  proposed  new  research  is  to  guide  and  to  clarify  an  optimum  iuturo 
development  path  lor  I  ,T  .  within  Hit'  military. 

Five  domains  of  research  are  suggested: 

(1)  Theory  of  instructional  strategies 

(2)  Curricula  and  courseware  development  for  applied  l.T. 

(3)  Course  evaluation  methods 

(d)  Equipment  and  software  aspects  of  applied  l.T, 

(5)  l.T.  benefit  and  cost  analysis. 

Future  funding  should  be  applied  to  these  categories  on  a  priority  basis 
according  to  the  ranking  in  the  above  list. 
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I.T.  research  centers  should  be  established  by  the  DoD,  or  present 
educational  facilities  should  be  improved  to  coordinate  research  activi¬ 
ties  and  accelerate  the  appH  cation  caret,  results  from  the  I.T. 

laboratories  to  the  Held.  It  is  recommended  that  a  basic  I.T  research 
center  he  established  and  that  i,  bo  accessible  by  each  military  service. 
Also,  each  military  service  should  enhance  its  present  operational  centers. 
The  basic  I.T.  research  center  should  not  be  operated  by  any  specific 
military  service. 


The  basic  research  center  should  be  organized  with  balanced  partici 
pation  of  all  three  services  in  mind.  Its  objectives  should  be: 

•  To  provide  a  loundat ion  ior  nourishing  basic  I.T.  research 

•  To  monitor  and  validate  the  future  benefits  of  I.T. 


•  To  maintain  an  optimum  path  for  the  expulsion  of  I.T. 
in  the  DoD. 

The  comer  should  port.,™  and  support  basic  research  and  bo  coBuiaant 
of  the  present  and  projected  stale  or  the  an  in  I.T.  „  should  also 
mahe  recommendations  and  facilitate  research  in  "blue-sky"  areas,  mule 
it  id  difficult  to  prow  "blue-sky"  research  effectively,  the  center 
Should  he  responsible  for  lde„lifyi„c  „„  research  opportunities  and  for 
local j  ii};  appropriate  funding  resources. 


The  center  should  be  available  as  a  breadboard  facility  for  outside 
researchers,  and  it  should  provide  the  focal  point  for  other  I.T.  work¬ 
shops.  Each  military  service  should  augment  its  existing  facilities  or 


tunctions  in  I.T.  development, 
facilities  as  an  I.T.  workshop 
'v f  l h  the  basic  center. 


and  each  should  designate  one  of  its 
or  communication  center  for  interfacing 
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1 1 1  BACKGROUND 


A .  Instructional  Techno  1 o  gy 

As  discussed  in  Section  II-A,  I.T.  combines  modern  technological 
devices  and  student  motivating  techniques  to  improve  the  instructional 
process.  One  of  the  key  nonhuman  resource  elements  of  I.T.  may  be  the 
computer. 

The  role  that  the  computer  plays  in  I.T.  is  growing.  Computer-based 
instruction  (CBI)  can  be  cither  computer-assisted  instruction  (CAI)  or 
computer  managed  instruction  (CMI).  This  report  uses  an  expanded  version  of 
of  the  Army's  definition2  of  CAI  and  CMI.  CAI  is  a  man-machine  interac¬ 
tion  in  which  most  of  the  teaching  functions  are  accomplished  by  using 
the  computer  in  direct  support  of  a  training  situation.  Both  training 
and  tutorial  logic  are  stored  by  the  teacher  or  curriculum  developer  in 
the  computer  memory.  The  techniques  used  in  CAI  include  individualized 
instruction  and  group  dynamics  for  simulation  and  gaming,  logic  problem 
solving,  computation,  tutorial  dialogue,  drill  and  practice,  and  others. 

CMI  is  an  overall  instructional  management  system  in  which  detailed  stu¬ 
dent  information,  complete  curriculum  data,  and  information  on  available 
resources  are  integrated  to  facilitate  the  development  of  individualized 
piograms,  to  revise  curriculum  content,  to  provide  the  necessary  counsel¬ 
ing  and  guidance,  and  thus,  to  bring  about  optimum  instructional  resources 
management. 

1 .  Bencf its 

Benefits  from  I.T.  vary  considerably,  and  no  simple  common 
element  identifies  all  potential  benefits.  Therefore,  for  the  purposes 
of  this  study,  these  benefits  have  been  divided  into  two  categories  of 
variables:  tangible  and  intangible  bear  fits.  No  value  rankings  are  implied 

by  the  order  in  which  the  two  classes  of  variables  are  listed  below. 
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(1)  Tangible  benefit  variables 


•  Cost  redact  ions 

•  Reductions  in  training  time 

•  Shortening  ol  the  time  needed  t o  revise  curricula 

•  Manpower  savings. 

(2)  Intangible  benefit  variables 

•  Increased  comprehension  and  performance  of  the 
st  udent 

•  Better  student  and  teacher  motivation 

•  More  effective  use  of  the  military  services 
as  training  institutions 

•  More  reenlistments  due  to  increased  training 
effectiveness  and  student  satisfaction 

•  More  options  for  the  training  commands. 

Tangible  benefits  can  he  translated  directly  to  dollar  savings. 
Intangible  benefits  are  more  difficult  to  quantify,  but  hot!)  classes  of 
variables  affect  training  requirements. 

2 .  Tra in ing  bi fecti vcncss 

'lhe  military  services  have  a  continuing  need  to  improve  l  ho 
effectiveness  of  their  training  methods.  Cost  and  manpower  economies 
arc  sought  at  every  level  in  t lie  Dob.  Although  training  is  becoming 
increasingly  complex,  savings  are  still  imperative.  While  the 

3 

sophistication  of  equipment  lias  dramatically  increased,  the  abilities 
of  men  coming  into  t ho  service  have  remained  constant.  These  problems 
will  intensify  as  the  services  shift  to  an  all-volunteer  force. 


♦The  impact  of  this  problem  was  accelerated  b\  Secretary  Laird's 
announcement  of  t lie  carl>  termination  of  the  draft  (January  1973). 
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The  services  are  now  beginning  to  shift  the  responsibility  for  <  rain¬ 
ing  to  smaller  units,  thus  increasing  the  need  for  improved  self-management 
and  learning  motivation.  At  the  level  of  the  small  training  unit,  this 
means  that  training  skills  and  aids  must  have  greater  transferability. 

Having  recognized  some  of  the  potential  benefits  and  requirements 
of  I.T.,  the  services  have  revised  their  training  programs  and  policies 
to  some  extont/1’6  However,  a  unified,  internally  consistent  program 
remains  to  br  set  forth  to  measure  the  benefits  and  effectiveness  of 
operational  programs  that  apply  the  results  of  I.T.  research. 


B, 


He s earcli  Objectives 


Military  Iralnini!  is  always  in  a  stale  of  revision  ami  mode rniaal ion. 
Training  requirements  channel  cost  reduction,  are  made.  New  training 
objectives  and  methods  of  evaluation  .are  n.ondUV  being  inirodneed.  New 

,  a  ,  it  i  c  vvidelv  assumed  and  already  obseived 

techniques  for  training  evolve. 

<  ii, at  r  t  i  =  in  important  factor  in  ibis  dynamic 
in  preliminary  experiments  that  I.T.  is  an  impoi  i 

field,  but  a  clear  rale  has  not  c, „ creed.  1'hls  study  was  undertaken  to 
help  place  I.T.  In  a  rational  perspective.  The  research  object 1.0.  were: 

(D  t..  c.l.lisl,  the  criteria  f-v  ..easurtm:  the  iaeneiits  and 
effect ivencss  of  I.T. 

•  To  determine  the  parameters  to  be  measured. 

•  To  identify  selected  areas  of  training  subjects  whole 
the  application  of  I.T.  would  readily  yield  benefits. 

(2)  To  study  the  development  of  I.T.  training  efforts  wilhiat 

the  services  and  to  assess  the  commonality  of  t  hose  ei' torts. 

•  To  relate  advantages  and  disadvantages  of  joint 
training  programs  using  I.T. 

•  To  determine  the  impact  ol  increased  and/or  decreased 
commo n  efforts. 
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I  .T. 


•  To  postulate  the  basis  for  training  policies  and  future 
research  programs. 

•  To  provide  new  research  objectives  that  will  establish 
a  foundation  for  realizing  further  benefits  from  1.1. 

The  recommendations  derived  from  this  study  arc  given  in  Section  VI, 
page  71. 

C .  Achieving  the  Oi  iectivcs 

To  achieve  the  research  objectives,  efforts  were  directed  primarily 
toward  measuring  potential  cost  and  manpower  savings  from  applied  I.T. 
This  was  accomplished  by  identifying  the  meaningful  parameters  of  I.L. 
and  developing  a  mathematical  model  based  upon  payoff  criteria.  Present 
military  training  programs  were  examined,  and  a  considerable  data  base 
was  developed.  The  methodology  used  is  discussed  in  Section  I\ . 

1).  'flic  Use  of  Inst  met  ional  Tee  It  no  logy  in  Training 

During  the  last  two  decades,  each  service  has  explored  the  possible 
contributions  of  I.T.  in  its  training  programs.  In  fact,  the  military 
pioneered  most  of  the  research  in  the  use  of  film,  videotape,  television 
and  computers  as  instructional  aids.  bach  of  the  services  lias  estab¬ 
lished  its  own  training  objectives  and  pass/fail  criteria,  and  cacti  Itas 
also  developed  its  own  evaluation  procedures  for  determining  t  lie  extent 
to  which  training  objectives  have  been  satisfied. 

•1 

Army  goals  are  based  on  t  lie  following  objectives: 

(1)  Training  will  be  performance  based. 

(2)  Techniques  of  training  will  focus  less  on  subject 
matter  and  more  on  functional  context. 
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CD  Absolute  rather  than  normative  criteria  .111  Ik-  used. 

(,,  T  as  tins  .111  be  pcrturmanco  oriented  and  measurement  .ill  be 

on  a  so  or  no-go  basis. 

(5)  Individualizing  methods  .ill  be  used  to  the  greatest 
extent  possible. 

<6)  immediate  ieedbaeh  .ill  be  provided  a,  the  training  site  to 
the  train-..,  the  trainer,  and  the  training  management. 

(7)  A  rigorous  duality  control  system  .ill  bo  used. 

The  Armv  instituted  this  system  because  ot  a  conviction, 

based  upon  a  good  deal  .1  evidence,  that  .be  lecture/ 

.  .law  mntg i  effective  method 
demonstration  technique  is  not 

4 

to  promote  learning. 

The  Air  Force,  using  concepts  ol  system  analysts,  evolved  its  o.n 

training  procedure-instructional  system  devol„p.«nt .  As  stated  by  an 

Air  Force  training  representative: 

"It  grew  out  ot  programmed  learning 

have  applied  extensively  over  time - 

job- relevancy  ot  training  an  ollliio-visual  devices 

It  calls  tor  the  eroat  tve  use  ot  those  a  [r0<>.  the 

in  manpower  requirements. 

■  lo,.  tn  those  of  the  Army.  Courses  arc 
These  goals  aid  objectives  are  similai 

,  .  ,f.VM  and  criterion-based  tests  arc 

tailored  to  fit  job  performance  objectives, 

,  a  •  i  i  lonrn  1  ho  IlCCCSSill'y  J Ol3 

us0d.  It  is  implied  that  every  student  ..11  learn 

i  hi  matching  the  man  and  the  job  moie 
skills.  In  fact,  training  is  aimed  at  matching 

6 

nrocisclv  than  in  the  past. 

P  '  .  .  which  give  special  consider- 

The  Navy  has  the  following  objectives  , 

7 

ation  to  shipboard  requirements: 
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(1)  To  develop  individualized  training  involving  specific 
learning  objectives  and  stringent  performance  evaluation. 

(2)  To  promote  simplification  of  the  learning  process  through 
increasing  hands-on  experience,  the  involvement  of  the 
learner,  and  t  lie  use  01  _ouiuelors.  This  intensifies  the 
need  for  well- integrated  training  aids,  materials,  and 
devices . 

(3)  To  develop  valid  performance  measures  of  individual  and 
team  proficiency  to  be  used  in  training  and  in  fleet 
ope  rat  ion . 

(4)  To  promote  shipboard  training  where  it  can  be  clearly 
demonstrated  as  the  most  productive  site  for  a  training 
program. 

(5)  To  develop  effective  ways  to  use  simple  synthetic  training 
devices  and  simulators  to  reduce  training  time  and  fleet 
training  cost,  and  to  increase  standardization  of  perfor¬ 
mance  throughout  the  Navy. 

All  throe  services  emphasize  the  educational  goals  of  individualized 
instruction  and  performance  testing.  As  a  result,  the  large  lectures 
familiar  in  both  military  and  nonmilitary  classrooms  are  no  longer  so 
pervasive.  The  instructional  strategies  most  frequently  employed  in 
realizing  these  revised  training  goals  are  self-paced  instruction 
(including  programmed  instruction),  peer  instruction,  on-the-job  train¬ 
ing,  and  simulation. 

Each  ol  these  innovative  i  list  ruct  ional  strategies  may  be  used  with 
a  variety  of  media.  Often,  one  or  more  strategies  arc  used  in  conjunction 


An  example  of  peer  Instruction  was  developed  by  the  HumRRo  (Human 
Resources  Research  Organization),  Division  3  and  applied  to  the  Army's 
Basie  Combat  Training. 
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with  one  or  more  types  of  media  or  equipment.  Perhaps  the  most  common 
combination  used  is  the  lecture  and  the  programmed  textbook.  The  lecture 
in  combination  with  television  is  not  uncommon,  and  CHI  is  used  lor  self- 
paced  instruction  and  simulation. 

Although  those  identifiable  separate  efforts  exist,  there  is  no 
explicit  policy  for  extensive  common  use  of  technology  within  the 
military  establishment.  However,  statements  of  general  training  policies 
often  refer  to  technological  advances  and  their  implied  uses  to  support 
t  raining. 

Policies  implicit  in  the  studios  initiated  b\  the  services  demon¬ 
strate  increased  concern  with  I.T.,  particularly  as  It  applies  to  tech¬ 
nical  training.**’**  ^  Some  interservice  conferences,  such  as  the  CAI 
"Shirtsleeve"  Conference  at  Fort  Bliss,  Texas,  in  1071  and  the  annual 
NS  1A  Application  of  Computer's  to  Training  (ACT)  Conferences,  are  examples 
of  cooperative  efforts  to  achieve  common  instructional  goals.  All  of  the 
services  arc  interested  in  exploring  t  lie  use  of  computers  ns  t  mining 
devices.  The  increasing  number  of  statements  advocating  the  use  oi 
computers  for  various  instructional  purposes  suggest  that  military  train¬ 
ing  administrators  believe  that  the  use  of  computers  can  upgrade  training. 
Financial  support  is  evident  from  the  backing  that  is  being  extended  to 
groups  doing  research  in  CD1,  In  these  research  el  forts,  emphasis  is 
often  placed  on  reducing  training  costs. 


^Examples  include  statements  made  by  the  Board  of  Dynamic  training,  CON ARC 
Task  Group  Report,  "Shirtsleeve"  Conference,  and  the  NSIA  AC  1  Conferences 

+ARPA ,  the  four  military  services,  and  NSF  have  supported  a  broad  range 
of  research  in  CBI.  The  studies  extend  from  large-scale  delivery  systems 
such  as  T  ICC  IT  and  PLATO,  to  CBI  languages  such  as  LOGO ,  to  the  use  of 
the  ARPANET  for  supporting  CBI  research,  to  fundamental  studies  in 
artificial  intelligence,  to  the  exploration  of  fundamental  pedagogical 
procedures  such  as  confidence  testing. 
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Training.  Costs 


1 . 

Training  costs  .'eproscnt  a  large  expenditure  in  the  DoD,  and 
they  constitute  an  area  in  which  I.T.  benefits  can  be  visibly  demonstrated. 
For  example,  the  DoD's  total  national  expenditure  on  military  training 
in  1975  is  projected  to  range  from  $5  to  $8  billion,  or  8  to  9  percent  of 
the  estimated  1975  military  budget.  If  advances  in  I.T.  were  to  realize 
a  15  percent  reduction  in  military  training  costs,  the  potential  savings 
would  amount  to  $9.75  to  $1.2  billion,  or  approximately  1.25  percent  of 
the  total  1975  Dol)  budget. 

Figure  1  shows  possible  trends  in  the  estimated  GNP,  estimated 
DoD  expenditures,  and  estimated  DoD  training  expenditures.  This  figure 
is  based  upon  a  factual  DoD  cost  history  and  an  average  annual  growth 
rate  of  1.2  percent  for  the  GNP.  The  data  sources  are  given  in  Table  1, 
which  presents  estimated  DoD  military  training  expenditures. 

Present  research  results  in  I.T.  in  both  the  military  and 
nonmilitary  indicate  wide  variations  in  potential  cost  savings.  However, 
most  of  the  results  imply  significant  cost  reductions  arc  possible. 
Appendix  A  summarizes  some  examples  of  recent  I.T.  research  and  cites 
author  conclusions  about  changes  in  costs. 

In  measuring  I.T.  costs,  there  is  considerable  disagreement 
duo  to  differing  assumptions  as  to  the  appropriate  components  of  such 
costs.  This  problem  occurred  repeatedly  during  the  research  program, 
and  it  is  discussed  in  some  detail  in  Section  IV.  Duo  to  those  disagioe- 
ments,  studies  on  the  costs  of  I.T.  usually  yield  large  amounts  of  non- 
comparable  cost  data.  This  is  most  noticeable  in  studies  compelling 
civilian  and  military  costs,  although  similar  problems  exist  in  intei- 
and  intrascrvicc  studies.  Cited  cost  reductions  resulting  from  the  use 
of  I.T.  in  civilian  educational  settings  most  often  rely  on  the  assumed 
combination  of  a  volume  of  users  and  efficient  use  of  I.T.  to  produce 
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BILLIONS  OF  DOLLARS 


Table  1 


ESTIMATED  DoD  Ml  LITAHY  TIMIN'  INC  EXPENDITURES 
1965-1975 

(In  Bill  ions  of  Dollars) 


DoD 


'isca  1 

Year 

DoD 

* 

Outlays 

GNP  + 

DoD 

Out  lays 

as  percent 

of  GNP 

Training, 
Medical,  and 
Other  General 
Personnel* 

DoD 

M  i  1  i  t  a  ry 
Training 
Act i vit ies 

1965 

$•17 .  I 

$68  1 . 9 

6.9r) 

$5 . 9 

•S  I  .  0 

1966 

55.4 

7  19.9 

7.3 

7.5 

5 .1 

1967 

68.3 

793.9 

8.6 

8.5 

5 , 8 

1968 

78.2 

865.0 

9.0 

9.1 

G.  1 

6.7 

1969 

1970 

1971 

1972 

78.  1 

79,0 

931.1 

976.5 

101 6. 8 

8.1 

8.0 

9.8 

1973 

197  1 

68-85 

1137 

6.0-7 .5 

8.6-10.7  ** 

5. 9-7. 3 

1975 

62-86 

123  1 

5. 0-7.0 

7 .8-10.8** 

5 . 3- 7 . 3 

Sourc-o  for  tho  1965  to  1970  data,  "The  1970  Budget  and  Defense  Program 
'  1970  71’  Clnrk  CUf  ford's  statement  to  the  House  Armed  Services 
Committee,  p.  157  (15  Januaiy  1969).  Data  for  1973  and  1975  were  esti¬ 
mated  by  assuming  that  DoD  outlays  m  these  years,  relative  to  the 
estimated  levels  of  the  GNP,  will  be  in  the  range  of  the  1965  percentage 

(IT  nee  H 


Source :  National  Income  and  Product  Accounts,  Survey  of  Current  Business  " 
Office  of  Business  Economics,  U.  S.  Department  of  Commerce.  Data  for  1973 
and  1975  were  estimated  by  assuming  an  average  annual  compound  rate  of 
growth  of  1.2  percent  after  1971. 


Source :  The  Fiscal  Year  1969-1973  Defense  Program, "  McNamara 
before  the  House  Armed  Services  Committee,  p.  211  (22  January 


s  statement 
1968)  . 


Source:  Senate  Hearings  Before  the  Subcommittee  of  the  Committee  on 
Appropriations,  FY  1972,  p.  161.  This  is  calculated  by  applying  the 
factor  68  percent  to  the  preceding  column.  The  operation  and  mainten¬ 
ance  appropriation  for  the  U.S.  Army  (actual  1970  fiscal  year  xperience) 
showed  that  68  percent  of  the  general  personnel  activities  category  was 
spent  on  military  training. 


**lhe  1973  and  1975  training,  medical 
t u re  data  are  estimated  by  applying 
live  years  of  data  (1965  to  1969). 


and  other  general  personnel  c.xpcndi  — 
a  factor  of  12.6  percent  based  on 
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cost  benefits. 


In  t  lie  results  for  t  lie  military,  short  on  ini'  of  training 
time  usually  accounts  for  the  principal  savings, 1 1-1 °  since  the  trainee 
receives  a  salary  during  his  training  period. 

Even  t hough  there  are  wide  cost  disparities  in  experimental 
systems  in  t lie  civil  sector,  it  is  possible  to  array  cost  ranges,***  as 
shown  in  Figure  2.  In  L Ills  figure,  cost  data  for  17  categories  ol 
instructional  media  are  ranked  from  highest  to  lowest  cost  per  student 
hour.  (The  direction  of  the  arrows  beneath  each  bar  in  Figure  2  indi¬ 
cates  Sill  s  projection  of  the  trend  in  cost  per  student  hour  during  t  lie 
1973  to  197j  period.)  Six  of  t  he  1/  instruct  ional  teclniic|ues  a  re  expected 
to  show  increases  in  the  cost  per  student  hour,  and  four  are  expected  to 
decrease.  The  left  side  of  Figure  2  shows  the  estimate*7  for  TV  CA1  in 
1975.  This  system  is  expected  to  lie  cost  competitive  with  all  other 
inst  met  ional  t  echnologies  . 

by  the  time  a  study  of  costs  in  the  C1U  area  has  been  published, 
enough  time  has  usually  elapsed  to  make  the  published  cost  per  student 
hour  an  overstatement  because  costs  are  decreasing.  For  example,  it  is 
now  possible  to  purchase  or  lease  a  CHI  system  using  a  minicomputer  and 
32  Teletype  terminals  within  an  estimated  cost  range  of  $0.15  to  $1.81 
per  student  hour.  This  example  is  for  rates  available  in  February  1973 
in  the  San  Francisco  Bay  Area. 

Vorc  detailed  information  on  various  systems  needs  to  be 
established  and  the  costing  procedures  analyzed  for  commonality  before 
such  systems  will  likely  on, joy  successful  and  wide  implementation. 


*l)ata  from  Computer  Curriculum  Corporation  of  Palo  Alto,  California. 

Tile  costs  depend  on  the  assumptions  made  as  to  the  number  of  years  for 
amortization  and  the  amount  of  student  use  per  year.  Costs  include 
hardware,  maintenance,  three  CA I  tutorial  packages  (without  audio), 
on-line  individual  and  class  reports,  workshops  for  instructors,  one 
teacher’s  manual  per  course,  and  unlimited  use  of  the  facility. 


COST  IN  DOLLARS  PER  STUDENT  HOUR 


1  On-Line  Computer  (CAI) 

2  Automated  Storage  and  Retrieval  of  Written 

and  Graphic  Materials 

3  Computerized  Programmed  Instruction 

4  Facsimile  Transmission  of  Documents  by 

Electronic  Circuits 

5  Small  Discussion  Group 

*>  Dial  Access  Instructional  TV 

7  Class  Lecture 
•  Books  and  Journals 
SOURCES  References  16  and  17 


9  Printed  Programmed  Instruction 
to  Standard  Audio-Visual  Aids 
it  Closed-Circuit  Live  Instructional  TV 

1 2  Closed-Circuit  Tape-Recoided  Instructional  TV 

13  Broadcast  Live  Instructional  TV 

*1 2 3 4 5 * 7  Closed -Circuit  Lectures  on  Public  Address  System 
ts  Broadcast  Tape-Recorded  Instructional  TV 
19  Dial  Access  Audio  Ta>>e  Recordings 
u  Educational  Radio 

S  A- 1  775-2 


FIGURE  2  RANGE  OF  COST  PER  STUDENT  HOUR  FOR  17  INSTRUCTIONAL 

TECHNIQUES  (19G9  TO  1970)  COMPARED  WITH  1975  CAI  ESTIMATE 
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The  emphr  is  on  a  set  of  agreed  definitions  for  costs  is  continually 

2,18,19 

being  stressed  by  all  sectors  using  C11I,  but  rs  yet  no  common 

20 

procedures,  such  as  those  outlined  by  Tickton,  have  been  implemented. 

2 .  Computer-Based  Inst  ruct ion 

The  extent  of  the  development  of  CBI  in  the  services  is  a 
function  of  available  facilities,  their  present  cost  effectiveness,  and 
the  technical  difficulties  associated  with  implementing  CBI  methods. 
However,  the  use  of  CBI  is  growing  nationwide  in  the  areas  of  administra¬ 
tive  data  processing  (e.g.,  scheduling),  and  educational  operations 
depend  upon  the  computer  (CMI)  in  many  school  systems.  CM I  services  are 
also  available  from  commercial  sources.  In  contrast,  CAI  has  only  re¬ 
cently  emerged  from  experimental  status.  When  the  computer  is  not  used 
in  an  interactive  CAI  mode,  it  may  still  ser"c  'n  inform  the  student  and 
teacher  by  CMI  techniques  which  booklet,  device,  or  equipment  should  be 
used  in  an  individualized  course  cf  instruction.  Thus,  potential  reduc¬ 
tions  in  course  time  can  be  realized  from  either  the  CMI  or  CAI  modes  of 
CBI . 

Reductions  of  30  percent  in  course  time  as  a  result  of  using 

CBI  arc  common.  In  the  area  of  electronics,  reductions  of  87  percent 
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have  been  reported  using  the  Lincoln  terminal  system.  Other  elect  ronic 

11-15,22 

courses  experiments  have  reported  reductions  of  20  to  55  percent. 
Significant  reductions  in  instructor  work  loads  and  course  overhead  costs 
are  also  implied. 

CBI  also  offers  some  valuable  intangible  benefits.  It  makes 
the  transition  from  school  training  to  field  training  possible  on  a  laige 
scale.  When  sophisticated  equipment  (e.g.,  radar,  sonar)  is  involved, 

CBI  offers  several  on-the-job  training  advantages,  including  the  following; 

•  The  trainee  can  learn  in  a  realistic  environment. 
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•  lie  can  learn  on  a«  •  ua  1  field  equipment,  which  can  be 
simulated  by  CI3I  >r  rare  situations,  as  well  as 
common  ones . 

•  C11I  exercises  the  field  equipment  in  important  modes 
and  thus  contributes  to  its  continuing  readiness. 

Operational  CHI  programs  (primarily  CAI)  can  be  found  in  12 
Ainu  schools  and  colleges.  At  the  Army  Logistics  Management  Center 
at  Fort  Lee,  Virginia,  a  Univac  computer  is  used  for  both  simulations  and 
tutorial  courses.  Practical  CAI  courses  are  available  through  the 
Medical  field  Service  School.  1  he  Army  War  College  s  ipports  a  course 
involving  such  areas  as  individual  research,  a  gross  national  product 
projection,  a  joint  force  budget  model,  and  quantitative  applications  to 
international  relations.  The  Army  Security  Agency  Training  Center  and 
School  oilers  a  large,  successful  CHI  course  in  Morse  code.  Since  19(3(3 
basic  electronics  courses  have  been  offered  via  CHI  at  Fort  Monmouth. 
Comput or- support ed  instruction  in  an  automated  environment  is  used  to 
train  supply  personnel  at  the  U.S.  Army  Quartermaster  School.  Kmphasis 
is  placed  on  CAI  drill  and  practice  at  the  U.S.  Amy  Infantry  School. 

The  Engineer  School  and  Air  Defense  School  mainly  use  CHI  for  problem 
solving. 

The  Navy  supports  CHI  projects  conducted  by  several  civilian 

f 

groups,  as  well  as  CHI  in-service  development.  “  “  Dr.  Richard  Atkinson 

of  Stanford  University  lias  developed  a  tutorial  course  to  teach  computer 
programming  that  iocuses  on  the  basics  of  learning  techniques.  The  late 
Dr.  Jaime  Carbonnc 11  of  Holl  Hera nek  and  Newman  was  performing  research 
witli  a  system  enabling  both  the  learner  and  the  computer  to  take  the 


♦Sources  include  conversations  with  Army  personnel  listed  in  Table  2  in 
Section  IV. 

+  Souices  include  conversations  with  N'aval  personnel  listed  in  Table  2. 
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initiative  in  a  dialogue.  Dr.  Seymour  Papert  of  MIT  is  supported  by  ARP  A 
(work  monitored  by  ON’R)  i'or  the  development  oi'  LOGO,  a  high-level  program¬ 
ming  language  useful  in  teaching  a  variety  oi  mathematical  concepts. 

Dr.  Joseph  Rigney  of  the  University  of  Southern  California  is  iocusing 
attention  on  performance  in  training  programs  involving  the  trouble¬ 
shooting  of  operating  equipment.  His  research  has  found  significant 
applications  in  CAI  tutorial  programs  involving  simulation  in  courses 
such  as  Radar  Repeater  (AN/SPA-6G)  or  Radio  Transceiver  (AN/URC-32) .  A 
Florida  State  University  contract  involves  on-line  intelligence  testing 
and  evaluation  and  several  studies  of  the  interaction  of  CAI  with  various 
psychological  states.  Juch  of  L  lie  CBI  pioneering  effort  in  the  Navy  has 
stemmed  from  the  work  of  Dr.  John  Ford  at  the  Naval  Personnel  and  Train¬ 
ing  Research  Laboratory  in  San  Diego. 

*26 

The  Air  Force  has  directed  its  CBI  funding  to  four  main  projects. 

The  main  operational  system  is  the  Computer-Directed  Training  System, 
which  provides  worldwide  on-the-job  training  in  a  variety  of  management 
functions  related  to  the  operation  and  use  of  computers.  The  Lincoln 
Terminal  System  developed  in  prototype  form  at  MIT  provides  both  audio 
and  complex  pictorial  capability;  the  prototype  system  is  now  undergoing 
developmental  tests  at  Keesler  AFB.  This  system  is  used  in  training  for 
basic  electricity  and  electronics.  In  addition,  special  provisions  have 
been  made  for  the  preparation  of  training  materials.  The  Advanced  Instruc¬ 
tional  System,  presently  undergoing  specification  and  bid  reviews  at 
Lowry  AFB,  involves  computer  control  of  a  wide  variety  of  media  to  allow 
for  sophisticated  individualized  instruction.  Initially,  the  system  is 
intended  to  support  research  on  computer-based  training,  as  well  as  to 
offer  three  technical  training  courses  for  up  to  2400  students.  The 
undergraduute  Pilot  Training  System,  now  being  developed  incrementally, 
will  use  CAI  in  the  teaching  of  pilot  skills.  Advanced  simulators  and 
simulation  techniques  will  be  used  in  this  program. 

♦Sources  include  conversations  with  Air  Force  personnel  listed  in  Table  2. 
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IV  COST  ANALYSIS 

This  section  identifies  the  data  base  used  in  the  research, 
develops  a  mathematical  model,  and  calculates  the  estimated  dollar  and 
manpower  savings  to  be  gained  from  the  application  of  I.T.  to  military 
training . 

A.  Data  Base 

The  data  base  consists  of  three  principal  segments: 

(1)  The  estimated  instructional  cost  per  hour  in  both  public 
and  military  instructional  sectors.  These  per  hour  costs 
are  based  upon  actual  experimental  implementation  of  various 
forms  of  I.T. 

(2)  The  expected  hardware,  software,  and  courseware  cost  trends 
for  I.T.  systems. 

(3)  Actual  annual  data  on  the  number  of  trainees  graduated  and  the 
training  costs  in  the  three  military  services. 

These  three  segments  of  data  are  the  smallest  set  of  inputs  necessary 
to  permit  the  calculation  of  dollar  and  manpower  savings  estimates 
from  the  mathematical  model. 

Together,  these  three  segments  constitute  an  extensive  data 
base  that  can  be  used  as  a  foundation  for  subsequent  interpretations, 
although  for  this  study  it  has  not  been  tapped  in  depth.  Tie  military 
manpower  and  training  cost  data  lias  been  collected  and  codec  ;  some  of 
it  is  now  stored  in  computerized  form  on  disk  files. 

1 .  Data  Survey 

Literature  searches  and  on-site  interviews  were  conducted 
to  initiate  the  data  collection  for  Segment  1  of  the  data  base. 

The  I.T.  group  at  SRI  utilized  its  contacts  with  the  civil  I.T. 


Preceding  page  blank 
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community  and  expanded  these  sources  through  direct  contacts  with  mili¬ 
tary  and  other  civilian  officials  to  identify  and  gain  access  to  cur¬ 
rent  information.  The  survey  identified  in  a  very  broad  scope  (not 
claimed  as  exhaustive)  pertinent  civilian  and  military  cost  elements 
associated  with  I.T.  Table  2  lists  some  of  the  individuals  contacted 
during  the  study. 

Segment  2,  equipment  and  courseware  costs,  is  discussed  in 
Section  IV-B,  This  segment  of  data  was  used  to  develop  parameters 
for  tile  cost  analysis  and  for  testing  the  sensitivity  of  the  cost 
model  to  changes  in  these  parameters. 

2.  Data  Collection 

To  obtain  the  Segment  3  information  for  representative 
costs  on  a  course-by-course  basis  emphasis  was  placed  on  finding  the 
original  data  as  actually  recorded  by  each  of  the  military  organizations. 
The  level  of  detail  sought  was  the  individual  technical  military  course 
with  all  of  its  associated  cost,  curriculum,  and  student  loading 
characteristics,  as  actually  pursued  in  a  given  time  frame.  The  time 
period  selected  was  the  most  recent  for  which  both  training  costs  and 
student  throughput  were  available  from  the  services.  In  all  cases, 
the  sources  of  data  sought  were  the  most  recently  available  (i.e., 
latest  publications)  course  descriptions  and  cost  compilations.  For 
examples,  see  Appendix  B. 

Sources  of  data  included  the  formal  school  catalogs,  the 
staffing  data,  and  school  throughput  data  (trainees  graduated)  as 
well  as  direct  interviews  with  personnel  involved  in  the  training 
activities  of  each  of  the  three  military  services.  Data  were  sought  at 
the  headquarters  and  staff  levels,  and  interviews  were  conducted  with 
both  training  management  and  training  clerical  personnel  responsible  for 
preparing  accounting  data  describing  technical  courses.  For  example, 


DATA  SURVEY  CONTACTS 


Mr.  Goldfine  Training  Office  of  the  6th  Army  San 
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on  ques- 


supcrvtsory  instructors  wore  queried  by  SRI  professional  staff 
tions  of  curricula,  student  motivation,  reasons  for  and  frequency  of 
changes  in  the  curricula,  and  the  mechanics  and  system  for  generating  th 
raw  data  on  school  throughput  (attendance,  attrition,  and  trainees  grad¬ 
uated)  at  the  classroom  level.  The  cost-per-trainee  information  was 
sought  and  found  in  the  financial  and  controllers'  offices  of  the  three 
military  services. 

Pei  tinent  s|)ecial  studies  on  costs  that  cover  the  presently 
used  definitions  and  methods  for  estimating  the  total  cost  to  train 
an  inductee  or  volunteer  to  achieve  a  given  skill  level  were  identified, 
and  in  some  cases,  the  raw  data  from  these  studies  were  used  in  the 
analysis . 

Examples  of  the  types  of  general  cost  information  on  training 
that  aie  currently  available  arc  the  'Military  Occupational  Specialty 
Tiaining  Cost  Handbooks,  Volumes  I  and  II,  published  by  the  U,S,  Army 
Field  Operating  Cost  Agency  in  November  and  December  1972,  respectively, 
Annual  Training  Time  and  Costs  for  Navy  Ratings  and  NECs,"  (fiscal 
1972  edition),  published  by  the  Bureau  of  Naval  Personnel,  Department 
of  the  Navy,  and  "U.S.  Air  Force  Program  Technical  Training,"  Volume 
I,  published  by  the  Headquarters  of  the  Air  Training  Command,  Randolph 
AFB,  Texas,  in  March  1972. 

Professional  military  and  civilian  personnel  of  the  three 
soi vice  organizations  who  are  concerned  with  the  common  curricula  and 
joint  cost  problems  were  interviewed  to  ascertain  their  perceptions  of 
trends  toward  commonalities  in  both  areas. 

3.  Summary  of  the  Data  Base 

The  established  data  base  was  placed  in  machine  readable 
form  by  keypunching  directly  from  the  source  documents  to  cards.  The 
data  base  includes  the  following  approximate  number  of  technical 
courses : 
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No .  o  r 


Service 

Courses 

Army 

2000 

Navy 

1200 

Air  Force 

1000 

Tins  data  base  is  now  in  a  form  amenable  to  computation  using  various 
mathematical  models.  Alternative  assumptions  from  those  discussed  below 
may  be  postulated  by  various  experts  in  the  specialty  fields  of  civil 
and  military  instructional  methods.  The  data  base  is  both  open-ended 
and  susceptible  to  rapid  refinements  and  further  iterations  because  of 
its  computerized  form. 


'1*  Sufficiency  and  Comparability  of  Data 

The  following  anomalies  exist  in  the  present  data  base: 

•  Cost  accounting  differences 

•  Catalog  inconsistencies 

•  Variations  in  training  methods  among  the  three  services 
(which  need  further  examination) 

•  A  lack  of  complete,  detailed  course  curricula  descriptions. 

Where  the  anomalies  were  apparent,  they  were  excluded  from  the  data 
sample  used  in  the  calculations.  For  the  purposes  or  this  macroanalysis, 
it  was  concluded  that  the  fine  structure  and  detail  in  the  complete 
course  curricula  would  not  significantly  change  the  results. 

Cost  accounting  varies  from  service  to  service,  and,  to  some 
extent,  within  a  given  service.  No  adjustments  were  made  in  the  data 
base  to  the  total  cost  per  graduate,  and  the  definitions  of  the  cost 
per  trainee  arc  those  given  by  each  military  service.  Thus,  costs  of 
equipment  used  in  a  given  training  course  may  or  may  not  be  reflected 
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iii  the  cost  per  graduate,  depending  on  the  specific  service's  accounting 
procedure.  This  will  not  degrade  the  computational  results  significantly, 
since  the  overall  inferences  from  this  study  arc  not  intended  to  have  a 
specific  course  orientation. 

Differences  in  military  training  catalogs  constitute  the 
second  class  of  anomalies.  Because  of  variations  in  interpretations  of 
MOS  (military  occupational  specialty),  AFSC  (Air  Force  specialty 
classi f ication) ,  and  NEC  (Naval  enlisted  classification),  it  is  often 
difficult  to  determine  when  courses  with  similar  descriptions 
designed  to  train  an  individual  for  a  given  specialty  have  enough 
commonality  to  permit  their  application  service-wide  with  the  aid  of 
new  I.T.  Til  is  anomaly  is  of  no  concern  to  the  cost  results  reported 
herein  because  all  samples  were  specific  to  each  service.  Thus,  this 
anomaly  also  impacts  only  the  commonality  aspect. 

In  some  cases,  courses  may  already  be  revised  for  programmed 
instructions,  or  they  may  include  other  aspects  of  I.T.  without  such 
effects  being  reflected  in  the  catalog  or  in  the  cost  and  throughput 
data  provided  by  the  services.  This  implies  that  cost  differentials 
projected  in  subsequent  sections  based  upon  1971  data  reflect, 
in  some  cases,  savings  estimated  by  the  model  that  may  have  already 
been  effected.  Again,  this  anomaly  is  of  little  concern  to  the  overall 
project,  since  the  results  are  not  intended  to  be  dependent  on  single 
courses . 

The  third  class  of  anomalies  is  due  to  variations  in  training 
methods.  For  example,  in  teaching  mathematics,  the  Navy  uses  classical 
mathema tical  educational  procedures,  the  Army  prefers  handbook 
techniques,  and  the  Air  Force  uses  a  combination  of  both  methods. 

These  differences  are  not  reflected  in  the  military  catalogs,  and 
readers  should  be  aware  of  this  type  of  inconsistency  in  the  data. 

Other  variations  in  tralning--e.g. , 


m-the-job  training--also  impact  the 


net  training  budget  and  I.T. 


savings,  but  they  arc  not  reflected  in 
this  study,  which  is  based  only  on  formal  technical  training.  These 
effects  arc  discussed  in  Section  \. 

In  cases  where  data  for  a  given  course  were  incomplete,  the 
course  was  excluded  from  the  data  sample  used  in  the  calculation. 
Nontechnical  courses  were  not  included  in  the  data  base,  but  evidence* 

suggests  that  the  yield  from  newly  applied  I.T.  for  these  courses  can 
also  be  substantial. 

5*  Sample  Data  for  Calculations 

The  logistic  problems  associated  with  analyzing  data  on  over 

‘1,00°  miUtary  t,,,ur8es  aiL>  subst  ani  ini  .  The  method  used  in  this  project 
was  to  select  sample  courses  from  each  of  three  training  commands.  The 
courses  are  representative  of  the  general  field  of  military  training, 
as  described  in  the  formal  school  catalog.  Courses  selected  represented 
a  diversity  of  curricula. 

Restrictive  requirements  affecting  these  selections  wore  that 
the  cost  and  manpower  data  base  be  complete  for  the  chosen  course  and 
Uiat  a  description  of  the  course  content  be  available.  The  latter 
requirement,  though  not  directly  related  to  cost,  is  necessary  to  the 
assessment  of  whether  the  application  of  I.T.  will  be  appropriate  for 
the  given  course.  It  allows  the  determination  of  present  levels  of 
I.T.  (in  some  cases)  and  provides  for  detenninlng  which  I.T.  media 
would  be  most  appropriate.  With  this  knowledge,  an  estimate  of  the 
form  ot  instructional  strategies,  technical  equipment,  and  other  I.T. 
parameters  can  bo  compared  with  known  conclusions  from  experimental 
or  prototype  courses. 


T 

Rc|x>rts  by  Richard  Atkinson,  Donald  Bitzer,  Duncan  Hanson,  Seymcur 
Paper t,  Patrick  Suppes,  and  others.  CONARC's  Task  Group  Report  on 
CAI  recommends  extension  ol  I.T.  application  to  nontechnical  courses,  as 
(Iocs  the  Shirtsleeve  '  Conference.  Also  see  "instructional  Technology 
in  the  Annul  Forces,"  bv  It.  Col.  II.  B.  Hitchens  (Ref.  23) . 
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Courses  with  zero  throughput  were  rejected  from  the  sample 
used  in  tin  ral  cula  l  ions  .  Courses  that  had  an  estimate  lor  f(CT)~  tin 
li  act  tonal  reduction  in  course  l  ime--ei|ua  1  to  zero  Here  also  excluded 
since  there  would  be  no  rationale  for  int reducing  I.T.  in  such  courses. 

The  total  number  of  courses  actually  used  in  the  calculations 
was  119,  drawn  from  each  of  the  services  as  follows: 


0 

c 

35 

Service 

Courses 

Air  Force 

30 

A  nil  y 

39 

Navy 

50 

To  t  a  1 

lit) 

The  Navy  data  sample  is  somewhat  larger  than  that  of  the  Army  and  Air 
lotce  because  the  data  sample  from  a  separate  manpower  study  (sponsored 
by  OMt)  was  also  used  as  an  input  to  the  analysis  for  this  report. 

1!.  Mathematical  Model  and  Payoll  Criteria 

Ilic  mathematical  model  used  considers  estimated  costs  for  developing 
I.f.  programs;  lor  obtaining,  maintaining,  and  operating  equipment;  and 
lor  creating  and  distributing  training  materials.  A  single  payoff  cri¬ 
terion,  a  reduction  in  course  time  that  yields  cost  and  manpower  savings, 
is  the  payoll  function  Unit  integrates  the  effects  of  13  critical  I.T. 
variabl es . 

1 .  Overview 

The  project  team  was  concerned  with  developing  useful  models 
and  identifying  significant  parameters.  Of  particular  concern  was  the 
identification  ul  a  value  function  that  would  be  quantifiable  and 
sell-converging,  so  that  each  iteration  of  the  model  would  tend  to 


improve  the  resolution  o!  tlie  benefit  evaluation.  From  a  qualitative 
point  of  view,  the  study  team  searched  tor  models  that  would  generate 


ranges  of  values  for  the  given  parameters  and  that  would  establish  cost 
levels  above  which  the  application  of  I.T.  would  yield  benefits. 

The  data  analysis  is  not  rigorous  in  a  mathematical  or 
statistical  sense.  Random  samples  were  not  selected,  and  the  size  of 
the  sample  was  small  compared  to  the  total  population.  Course 
selection  for  the  sample  calculations  was  based  instead  ii|X)n  completeness 
of  data.  Relevant  I.T.  variables  were  identified,  and  attempts  were 
made  to  integrate  their  individual  effects  in  a  single  payoff  function. 
Given  the  selected  payoff  value  (fractional  reduction  in  course  time), 
exemplary  calculations  for  cost  and  manpower  savings  were  performed. 

The  benefit  was  quantified  in  terms  of  annual  military  cost  and 
manpower  savings. 

The  size  of  the  sample  courses  selected  is  a  minor  restriction 
on  the  results  of  the  analysis;  it  simply  means  that  the  data  base 
has  noc  been  exhausted.  The  sample  is  of  sufficient  size  to  recognize 
trends,  to  identify  cost  differentials,  to  relate  the  effects  of 
different  types  of  cost  factors  on  net  cost,  and  to  compare  the  results 
of  different  types  of  I.T.  innovations  as  reflected  by  the  significant 
variables . 

Although  the  present  study  docs  not  extend  the  quantitative 
implications  of  these  results  to  the  entire  field  of  training  in  the 
military  services,  it  provides  the  model  and  the  mechanism  for  a  more 
comprehensive  analysis  that  would  provide  estimates  of  the  dollar 
and  manpower  savings  expected  for  each  of  the  three  military  services. 

The  results  make  it  possible  to  determine  quantitative  differences 
in  the  adaptability  of  military  training  courses  to  I.T.  and  to  determine 
where  further  analysis  and  innovation  arc  justified  and  where  they  are 
not.  The  calculations  were  primarily  intended  to  serve  as  background 


for  identifying  future  research  goals  in  addition  to  demonstrating  a 
method  to  quantify  potential  cost  and  manpower  savings  from  applied 

I.T. 

2 .  The  Payoff  Criterion 

A  single  parameter,  defined  as  the  fractional  reduction 
in  course  time,  and  expressed  symbolically  as  5(CT),  was  selected 
as  the  basic  criterion  upon  which  to  base  cost  and  manpower  savings. 

The  cost  calculations  were  based  upon  values  of  S(CT)  that  could  be 
achieved  by  the  introduction  of  I.T,  while  maintaining  the  stated 
course  objectives  and  pass/fail  criteria.  The  calculations  were 
based  upon  estimated  values  of  ‘-(CT)  for  each  course  sampled.  These 
estimated  values  were,  in  turn,  determined  by  relating  a  set  of  13 
signi  icant  I.T.  variables  to  each  course  studied.  For  each  course 
studied,  an  optimal  mixture  of  the  I.T.  variables  was  postulated. 

The  model  assumes  a  maximum  value  of  5(CT)  of  0.5,  The  procedures 
used  for  estimating  6 (CT)  arc  discussed  in  subsequent  sections. 

3.  Significant  Variables 

The  13  variables  identified  that  contribute  to  savings 
in  training  time  achieved  by  the  application  of  I.T.  emerged  from  a 
study  of  a  wide  range  of  courses  to  which  I.T.  has  been  applied,  as 
well  as  technical  training  courses  presently  offered  in  the  services. 

Tlie  list  was  continually  modified  and  refined  as  new  data  were  examined. 
Additional  refinements  or  modified  interpretations  may  be  needed  in 
the  future  to  cover  sucli  situations  as  a  second  application  of  I.T. 
to  a  given  course. 

The  13  variables  were  divided  into  four  general  categories: 

•  Instructional  strategics 

•  Course  content 
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•  Course  conditions, 
a .  Instructional  Strategics 

The  four  variables  related  to  instructional  strategics 
arc  individualization,  inefficiencies,  immediate  feedback,  and 
evaluation.  They  are  briefly  described  below. 

Individualization — Many  courses  are  written  to 
allow  the  individual  student  to  proceed  through 
the  material  at  his  own  pace.  Some  courses 
also  individualize  the  content  of  a  course,  e.g., 
additional  background  material  may  be  provided  for 
si  I  acted  students,  students  may  be  given  varying 
amounts  of  practice  when  learning  a  new  skill,  and 
advanced  material  may  be  provided  for  certain 
s indents . 

•  I  net f iciencics--Each  time  a  course  is  revised,  it 
is  likely  to  produce  savings  in  course  time  by 
culling  out  repetitive  material,  by  clarifying 
explanations  that  proved  difficult  for  students, 
and  by  removing  outright  errors.  Application  of 
certain  forms  of  I.T.  (e.g.,  CBI)  is  exceptionally 
valuable  when  removing  such  inefficiencies.  For 
example,  the  nature  of  CBI  forces  the  curriculum 
designer  to  be  explicit  about  the  goals  of  the 
course  and  any  performance  measures  involved. 

•  Immediate  Feedback — This  is  defined  as  a  prompt 
response  to  the  student  that  gives  him  an  assess¬ 
ment  and  evaluation  of  his  input.  When  course 
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curricula  can  exploit  immediate  feedback  to  benefit 
the  student,  the  amount  of  time  saved  by  implementing 
I.T,  (CB1,  in  particular)  can  be  significant.  For 
most  courses,  immediate  feedback  to  the  student 
is  extremely  important.  Its  major  benefit  to  the 
student  is  that  it  provides  the  basis  for  on-the- 
spot,  ongoing  self-evaluation.  The  choice  of  the 
type  of  I.T,  to  be  used  is  affected  by  the  perceived 
desirability  of  immediate  feedback. 

*  Evaluation — Ongoing  evaluation  saves  testing  time, 
helps  pinpoint  and  clarity  misunderstandings  that 
could  lead  to  the  need  for  reniti  1  instruction, 
and  shortens  course  time  by  enabling  a  trainee  to 
finish  the  course  as  soon  as  lie  uemons trates  the 
desired  performance.  Evaluation  results  can  also 
aid  the  curriculum  designer  in  course*  revisions. 

b .  Course  Content 

The  four  variables  most  closely  associated  with  course 
tent  arc  commonality,  nonverbal  representation,  teaching  algorithm, 
stability.  They  arc  described  below. 

*  Communal i t y  Time  savings  in  the  preparation  and 
levision  of  curricula  exist  if  commonality  can  be 
found  in  one  or  more  courses.  The  degrees  of 
commonality  vary  greatly.  For  example,  courses 
may  be  identical,  at  least  in  part;  they*  may  use 
the  same  teaching  or  instructional  logic;  or  they 
may  use  the  same  equipment,  such  as,  visual  displays. 

*  Nonverbal  Representation- -Potential  savings  are 
found  in  the  application  of  I.T.  to  largely  nonverbal 
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courses  that  deal  with  motor,  graphic,  or  symbol 
manipulation  rather  than  linguistic  manipulation. 

It  is  difficult  to  adapt  nonverbal  technical 
training  to  the  traditional  lecture,  and  much  on- 
the-job  training  retains  a  high  verbal  component. 
Applying  selected  forms  of  I.T.  to  nonverbal  training 
reinforces  the  desired  visual/nonverbal  component 
while  allowing  student  participation  and  interaction 
with  tlie  teaching  strategies. 

•  Teaching  Algor i thin- -Some  courses  iiave  sufficient 
logical  structure  and  content  to  enable  them  to  be 
expressed  in  algorithmic  form.  Such  courses  will 
facilitate  the  use  of  certain  forms  of  I.T,,  such 
as  CDI. 

•  Stability — A  course  is  defined  as  stable  if  it  is 
relatively  immune  to  changes  in  its  basic  curriculum 
and  structure.  Courses  with  low  stability  are  usually 
more  easily  changed  in  an  I.T,  than  a  traditional 
setting.  Well-designed,  modular  CBI  courses  can  be 
easily  revised  to  reflect  modified  goals  or  a  change 
in  tlie  instructor's  viewpoint,  and  such  revisions 

are  less  costly  than  those  for  traditional  courses. 

Tlie  distribution  of  revisions  can  be  facilitated 
by  nailing  cassette  tapes  or  by  using  a  computer 
ne twork. 

c .  Course  Structures 

Three  types  of  training  course  structures  are  simulation, 
problem  solving,  and  on-the-job  training: 
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Simulation — Simulation  approaches  may  involve  a 
wide  variety  of  media,  and  they  may  be  designed 
for  use  by  an  individual  or  a  group  or  both.  For 
example,  the  trainee  pilot  can  practice  landing  in 
a  safe  situation,  without  the  possible  loss  of 
valuable  equipnent.  Strategics  can  be  examined 
through  the  use  of  simulated  war  games.  Weapons 
can  be  tested  in  a  variety  of  hypothetical  situations 
before  field  use.  Computers  are  being  used  to  "stim¬ 
ulate"  equipment  to  perform  in  an  unlikely  fashion 
so  that  trainees  can  practice  troubleshooting  using 
simulated  malfunctions.  Computers  are  well  adapted 
to  simulation  techniques  because  programmed  situations 
can  be  easily  duplicated,  updated,  and  calibrated. 

Problem  Solving — Certain  courses  deal  with  the 
acquisition  of  problem-solving  skills  rather  than 
the  acquisition  of  factual  information.  I.T.  in 
the  form  of  CBI  offers  several  advantages  over 
traditional  approaches  to  problem  solving.  The 
computer's  problem-solving  capabilities  can  serve 
the  student  on  an  ongoing  basis,  and  the  computer 
can  be  used  to  generate  problems  suitable  for 
each  student.  Most  CBI  courses  involving  problem 
solving  show  significant  time  savings  over  courses 
using  a  traditional  classroom  approach,  primarily 
because  oi  the  ability  of  CBI  to  analyze  portions 
of  tlie  problem-solving  processes. 

On-thc -Job  Training — The  amount  of  on-the-job 
training  (OJT)  that  is  desirable  in  a  course 
affects  possible  savings  in  course  time.  OJT  is 
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closely  related  to  simulation,  insofar  as 
simulation  may  be  viewed  as  an  attempt  to  replicate 
an  OJT  situation.  Some  OJT  procedures  strongly 
interact  with  a  number  of  other  variables,  such  as 
individualization. 

d.  Course  Conditions 

The  final  two  variables — proximity  and  heterogeneity — 
are  best  described  in  relation  to  course  conditions. 

•  Proximity- -Proximity  is  defined  as  the  proportion 
of  a  course  that  meets  in  the  classroom  rather 
than  on  the  training  field.  Proximity  influences 
(1)  the  amount  of  time  saved  through  the  application 
of  I.T.  and  (2)  what  type  of  I.T.,  if  any,  would  be 
most  useful. 

•  Heterogeneity- -In  a  traditional  classroom,  an 
instructor  usually  finds  it  extremely  difficult 
to  provide  relevant  instruction  for  a  class  whose 
preparation  and  abilities  vary  widely.  In  most  well- 
developed  I.T.  courses,  the  trainee  has  the  total 
attention  of  the  "instructor"  in  a  course  geared 

to  his  individual  needs. 

e .  Interrelationships  Among  the  Variables 

Table  3  indicates  some  perceived  areas  of  interaction 
among  the  variables.  Most  such  interrelationships  are  highly  complex 
and  course-dependent.  For  example,  individualization  will  usually 
save  course  time,  but  if  a  course  already  uses  extensive  methods  of 
self-paced  instruction,  further  possible  savings  through  the  use  of 
CAI  may  be  marginal.  However,  if  the  self-paced  course  yields  immediate 
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Table  3 


INTERRELATIONSHIPS  AMONG  THE  VARIABLES 
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feedback,  adding  both  individualization  and  feedback  may  create 
additional  time  savings. 


The  interaction  between  OJT  and  simulation  is  similarly 
complex,  and  the  decision  to  use  one  or  the  other  depends  on  the  specific 
goals  of  a  given  course.  Stability  and  commonality  also  interact; 
suppose  two  courses  have  identical  content,  but  one  is  highly  stable 
and  one  is  unstable.  The  stability  factor  strongly  influences  any 
interpretation  of  commonality. 

4 .  Estimation  of  5(CT) 

During  this  research,  the  13  variables  served  as  a  checklist 
in  the  judgmental  assignment  of  adaptability  of  a  given  course  to  further 
applications  of  I.T. 

Ranges  for  6(CT)  from  actual  experimental  results  demonstrate 
course  time  savings  of  20  to  70  percent.  The  average  is  approximately 
40  percent.  Examples  of  6(CT)  from  the  literature  served  to  relate 
the  course  under  study  to  a  set  of  calibration  data.*  In  each  case, 
a  best  mix  of  I.T,  strategies  was  postulated  for  estimating  the  reduced 
time  in  training  expected  for  a  trainee  to  reach  a  level  of  achievement 
identical  to  that  before  the  new  technology  was  applied. 

Fifty  percent  or  more  of  advanced  technical  training  in  the 
military  services  is  OJT,  for  which  no  curricula  descriptions  are  avail¬ 
able.  Therefore,  savings  derived  from  estimates  of  S(CT)  (based  only  on 
existing  written  curricula)  are  lower  than  the  total  savings  expected  if 
the  proposed  applications  of  I.T.  were  implemented  both  for  formal  school¬ 
ing  and  OJT  courses. 


Representative  experimental  values  of  5(CT)  used  as  calibration 
data  in  this  study  are  described  in  Appendix  D. 
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Ranges  of  values  were  established  for  6(CT) .  They  are 
commensurate  with  the  resolution  of  the  input  data.  They  permit  the 
classification  of  a  known  course  description  into  a  category  of 
adaptability  to  I  ,T .  Although  the  use  of  wide  ranges  for  6 (CT)  at  the 
microlcvcl  or  course  level  could  benefit  from  greater  quantitative 
pi  ec is  ion,  at  macrolevel  (estimating  total  savings  in  dollars  and  man¬ 
power  for  the  DoD) ,  it  was  not  found  necessary.  Table  4  lists  each 
category  for  (CT)  in  Roman  numerals,  the  corresponding  range  of  values, 
and  the  actual  values  used  in  the  calculations. 

Categories  of  S(CT)  for  each  course  analyzed  were  determined 
by  first  estimating  an  appropriate  value  of  5(CT) .  If,  for  a  specific 
course,  a  value  of  0.32  was  selected,  the  I .T .  Adaptability  Category 
would  be  III  and  the  actual  value  used  in  the  calculation  would  be  0.3. 


Table  4 

CATEGORIES  OF  6(CT) 


I.T.  Adaptability 
Category 

Range  of  Values 

Value  Used 

in  Calculation 

I 

0.00  -  0.15 

0.1 

II 

0.15  -  0.25 

0.2 

III 

0.25  -  0.35 

0.3 

IV 

0.35  -  0,45 

0.4 

V 

0.15  -  1.0 

0.5 

(maximum) 

Analytic  formulation  of  the  13  variables  relating  to  5  (CT) 
is  premature  now.  However,  the  thought  behind  each  estimation  of 
(CT)  as  it  relates  to  each  of  the  13  variables  can  be  exemplified  in 
some  detail,  as  shown  in  the  two  examples  discussed  below. 


a. 


Example  1 


Perhaps  one  of  the  best-known  series  of  military  CAI 
experiments  is  the  one  conducted  in  the  late  1960s  and  early  1970s  by 
the  Naval  Personnel  and  Training  Research  Laboratory  at  the  Basic 
Electricity/Electronics  "P"  School  in  San  Diego.11’12’22  As  an  example 
of  the  SRI  approach  used  to  estimate  6(CT),  consider  the  Inductance 
module  of  the  curriculum,  discussed  b\  Hurlock. 

Prior  to  applying  CAI  to  the  course,  the  module  on 
inductance  comprised  17  hours  of  instruction,  consisting  "primarily 
of  lectures,  question  and  answer  discussion  periods,  drill  and  piactici 
sessions,  assignment  sheets,  demonstrations,  and  short  laboratory 
exercises. ”12  Presentation  of  the  course  content  was  appropriate 
for  introducing  the  student  with  little  background  to  standard  concepts 
involved  in  inductance.  The  following  evaluation  relates  the  module 
to  the  13  variables: 


•  Teaching  strategics 

Individualization  is  seen  as  very  important 
in  such  a  course,  where  there  is  likely  to 
be  a  great  diversity  of  ability  and  interests. 

Inefficiencies  in  such  a  course  arc  likely  to 
be  high,  since  lectures  and  verbal  discussions 
are  relied  upon  to  teach  many  concepts  having 
a  high  nonverbal  component. 

Immediate  feedback  is  applicable  in  several 
important  areas  in  this  course,  mainly  recognition 
and  definition  of  terms  and  problem  solving. 
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Evaluation  is  particularly  useful  in  such  a 
course,  when  detailed  information  can  further 
individualize  the  course  and  prevent  the  need 
for  remediation. 

Course  content 

Commonality  of  a  topic,  such  as  inductance,  is 
hit'll.  It  is  found  in  a  number  of  specialities 
in  all  three  services. 

-  Nonverbal  Representation  is  common  in  this 
subject  area,  since  much  of  the  content  is 
easily  presented  in  diagrams  or  symbolic  form. 

Teaching  algorithms  are  standard  conventions 
for  teaching  basic  electronics  theory. 

Stability  of  such  a  course  is  high,  since  few 
changes  in  content,  goals,  or  equipment  occur. 

Course  structure 

Simulation  is  easily  and  usefully  done  using 
"real"  laboratory  equipment  that  is  both 
inexpensive  and  easily  managed. 

Problem  solving  is  a  very  important  part  of 
the  course,  since  it  is  a  major  route  to  a  good 
grasp  of  tile  subject. 

OJT  is  low  for  such  an  introductory  course  in 
electronics.  However,  the  use  of  "real" 


equipment  in  a  sense  partially  simulates  OJT. 


Course  conditions 


Proximity  to  the  classroom  is  high  for  sucli  a 
course.  No  field  experience  is  needed  at 
such  an  elementary  level. 

Heterogeneity  is  probably  high.  Students  can 
be  expected  to  vary  widely  in  background, 
ability,  and  interests. 

The  above  considerations  lead  to  the  conclusion  that  the 

application  of  I.T.  to  this  course — in  a  form  that  promotes  individualization, 

immediate  feedback,  simulation,  problem  solving,  and  ongoing  evaluation — 

would  result  in  significant  savings  in  course  time.  Thus,  the  estimated 

value  of  6(CT)  is  0.5  (Category  V).  This  value  of  6(CT)  corresponds  well 

with  the  experimental  results  of  18.5%  reduction  in  course  time  as 

12 

reported  by  Hurlock  in  1971. 
b.  Example  2 

The  second  example  is  the  Navy's  current  Basic  Electricity 

and  Electronics  Course,  identified  as  Course  A-100-00I0.  This  course 

12 

was  developed  as  a  result  of  such  I.T.  experiments  as  Hurlock  applied 
to  the  inductance  module.  This  course  already  incorporates  an  individ¬ 
ualized  learning  system.  It  is  assumed  that  a  second  application  of  I.T. 
to  the  course  will  result  in  lower  time  savings  than  the  original  appli- 
ca  t i on  . 

The  inductance  module  discussed  in  Example  i  is  a  sub¬ 
set  of  Course  A-100-0010,  and  the  majority  of  considerations  regarding 
the  relationship  between  the  course  and  the  6(CT)  variables  remain 
the  same.  However,  those  relating  to  individualization,  inefficiencies, 
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and  immediate  feedback  are  somewhat  different.  In  particular,  the 
individual i /.at ion  variable  contributes  less  to  time  savings,  since  the 
course  has  already  been  adapted  to  meet  an  individual's  needs.  Some 
additional  individualization  is  assumed  to  be  possible,  however,  because 
of  new  technology  and/or  research  results  on  instructional  strategies. 
Similarly,  since  I.T.  has  been  applied  to  the  course,  it  is  assumed 
that  ine f f iciencics  have  been  greatly  reduced  in  the  course.  limned i ate 
feedback  remains  an  important  factor  in  the  course  to  enhance  the 
student's  control  over  his  rate  of  progress  in  this  highly  individualized 
course . 

Those  considerations  led  to  an  estimate  of  S(CT)  as 
0.1  (Category  I)  for  this  individualized  basic  electricity  and 
electronics  course. 

C .  Cost  and  Manpower  Savings  Calculations 
1 .  Variables  and  Parameters 

Cost  and  manpower  savings  for  the  sample  courses  were  calcu¬ 
lated  using  the  estimated  6(CT)s  as  independent  variables.  Other 
significant  variables,  their  mathematical  representation,  and  their 
source,  are  indicated  in  Table  5. 


Table  5 

VARIABLES  USED  IN  THE  CALCULATIONS 


The  model  also  assumed  two  additional 


parameters:  (1)  equipment 


costs,  represented  symbolically  as  and  expressed  in  terms  of  dollars 

per  terminal  hour,  and  (2)  courseware  costs,  represented  as  C  and  ex- 

C 

Pressed  in  dollars  per  courseware  hour.  Both  C  and  C  are  represented 

C  h 

in  t lie  mathematical  model  in  terms  oi  amortized  annual  costs.  This  is 

consistent  with  common  practice  with  respect  to  C  ;  however,  it  may  he 

i:  ’ 

confusing  with  respect  to  courseware  costs.  The  following  example  should 
clarify  the  terms. 


Assume,  for  example,  that  the  total  cost  incurred  in  preparing 
one  hour  of  I.T.  courseware  material  is  $2000.  Assume  further  that  this 
material  must  be  completely  replaced  in  five  years  and  that  it  has  no 
residual  value.  Amortizing  this  cost  over  five  years  yields  an  annual 
courseware  cost  of  $400  ($2000/5)  per  hour.  The  mathematical  model  uses 
the  amortized  annual  cost,  $100  per  hour,  rather  than  the  total  cost  of 
$2000  per  hour,  and  the  results  are  calculated  on  an  annual  basis. 


In  estimating  total  I.T,  costs  lor  each  course,  it  was  neces¬ 
sary  to  estimate  the  on-line  factor,  o  ,  the  proportion  of  course  time 
during  which  equipment  and  courseware  would  be  in  use.  Tor  courses  with 
a  large  involvement  of  I.T.  and  a  large  estimated  6(CT),  there  is  a  rela¬ 
tively  heavy  use  of  equipment  and  courseware,  and  would  be  large.  On 

the  other  hand,  where  I.T.  is  of  little  use,  both  6(CT)  and  O  would  be 

T 

low.  For  this  study,  0,f  and  6(CT)  were  assumed  to  be  linearly  related: 

0,f  =  5(CT)  +  0.25  . 


°T  is  a  fractional,  dimensionless  term.  This  formula 
from  a  minimum  of  0.25  when  the  addition  of  I.T.  has 
to  0.75  which  corresponds  to  the  upper  limit  of  I.T. 
to  be  desirable.  Refer  to  Table  1  and  Its  associated 
the  range  of  e (CT)  is  presented. 


allows  0  to  range 
very  little  benefit 
that  is  considered 
discussion,  where 


> 


0T  has  a  major  impact  on  the  calculation  of  the  cost  of  imple¬ 
menting  I.T.  in  a  course  because  it  determines  both  the  number  of  class 
hours  that  are  devoted  to  equipment  and  the  required  number  of  lecture 

hours  that  must  be  converted  to  courseware  hours.  This  is  discussed  in 
greater  detail  below. 

2.  Equations 

The  four  principal  equations  arc  those  for: 

•  Total  cost. 

5  C°St  °f  imPlomenting  I.T.,  as  measured  in  equipment 

and  courseware. 

•  Cost  savings. 

•  Manpower  savings. 

The  annual  total  cost,  c?,  is  determined  by  multiply!,,,,  the 

throughput  Tp  (number  o!  graduates)  by  the  coat  per  graduate  C  1  o 

G’  > 


CT  =  TP  *  CG 

The  annual  cost  of  implementing  I.T..  c^,  is  a  function 


(1) 


o  equipment  costs,  course  time,  courseware  costs,  on-line  factor,  and 
throughpu  t : 


CI»1P  -  40  X  t  X  [l  -  «  <CT)j  x  0T  (CI;  X  Tp  *  cc) 


(2) 


In  Eq.  (2)  the  first  three  factors  represent  total  hours  per 
year  devoted  to  I.T.  training,  and  the  expression  in  parentheses  represe 
the  annual  cost  per  hour  of  I.T.  equipment  and  courseware.  A  machine  lo 

tng  of  one  trainee  per  terminal  is  assumed.  A  dimensional  analysis  may 
help  clarify  the  equation: 
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c 


IMP 


hours  /cciu  Lpment  cost 

-  v  weeks  ,  -  y  annual  terminal  loading 

week  (terminal-hour 


+ 


annua  I 


courseware  cost 
hour 


(2a) 


This  reduces  to: 


C 


IMP 


total  classroom  hours  x 


annual  courseware  cos  t 

.  ,  .  — - — - - 

hour 


^annual 
)  • 


equipment 

hour 


cost 


(2b) 


The  annual  cost  savings,  s  ,  are  determined  by  applying  the 
fractional  reduction  5(CT)  to  total  costs  and  then  subtracting  the  cost 
of  implementing  I.T.,  i.e., 


S 

C 


6(CT)  X  CT 


(3) 


The  annual  manpower  savings,  S  ,,  are  determined  by  applying 

.  _  __ . _  [,1 

the  fractional  reduction  in  course  time,  t(CT),  to  course  throughput: 

S  =  1.1  X  T  x  f(CT)  x  t/52  .  (d) 

M  P 


The  coefficient  1.1  used  in  Ecj .  (4)  reflects  estimated  instructor  man- 

* 

power  savings  and  assumes  an  instructor-to-trainee  ratio  of  1:10. 

3  .  Parameters  and  Variations 

To  determine  the  sensitivity  of  the  results  to  the  variables 
in  the  savings  model, three  parameters  were  varied:  6(CT) ,  equipment 
costs,  and  courseware  costs.  Each  was  varied  discretely  over  a  range,  os 


* 

SRI  estimate. 
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summarized 

Figures  3, 
delineated 
Figures  G, 


in  Table  6.  The  effects  of  varying  MCT)  arc  shown  in 

and  5‘  ThC  C,fCCtH  °f  vary ing  CE  and  Cc  with  savings 
m  both  value  ($)  and  real  (manpower)  terms  are  shown  in 
7,  and  8. 


Table  6 

PARAMETER  VARIATION  VALUES 


Parameter 

Range  | 

Equipment  costs  (I.T. 
Delivery  Cost /Hour) 

$0,  50-$4. 00  per  terminal  hour 

Total  courseware 
production  costs 

S0-$10,000  per  course  hour* 

£(CT) 

*  “ 

Estimated  value  of  5(CT)  6(CT)-0  1 

t (CT) -0 .2  '  ’ 

- - - - 

*  - ■ - . - - 

Corresponds  to  annual  courseware  production  costs  of  0 
$2,000  based  upon  a  five-year  amortization. 


th°  Sclcctio"  of  the  range  of  equipment  costs,  the  cos 
or  computer  hardware  was  assumed  to  l,c  larger  than  that  or  any  othc 

type  or  I.T.  equipment.  Thus,  the  selected  equipment  cost  range 

reriects  CBI  hardware  costs.  For  courses  that  use  other  types  or  I 

hardware  (c.g.,  audio-visual  devices)  thn  i  i  * 

uevices;  ,  the  calculations  are  somewha' 

conservative . 


In  the  assignment  of  categories  of  5(CT) ,  a 
Students  per  terminal  was  assumed,  using  alphanumeric 
assumption  impacts  only  on  equipment  costs.  As  shown 
that  can  he  realised  through  the  use  of  I.T.  are  less 
hardware  costs  than  other  parameters. 


1:1  ratio  of 
terminals.  This 
below,  savings 
sensitive  to 
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TRAINING  COST  SAVINGS  TRAINING  COST  SAVINGS 

millions  ol  dollars  m.ll»n»  ol  dollars 


dollars/course  hour 

TOTAL  TRAINING  COST  FOR  SAMPLE  *  $75,000,000 

T  A-7 1 0582-42 

FIGURE  3  EFFECTS  ON  DOLLAR  SAVINGS  (TRAINING  COSTS)  FOR  SAMPLED  U.S.  ARMY 
TRAINING  COURSES  FROM  VARYING  THE  VALUE  OF  6(CT)  FOR  4  I.T. 
DELIVERY  COSTS  (CF> 


30 


FIGURE 


ANNUAL  COURSE  DEVELOPMENT  COST  (C  ! 
dollars  per  course  hour 


TA-710582  <J3 


effects  on  dollar  and  manpower  savings  for  sampled  us  army 

TRAINING  COURSES  FROM  VARYING  I.T.  DELIVERY  COST  (C 7 AND  COURSE 
development  COST  (Cc>  for  CONSTANT  *(CT)  e 


TRAINING  TIME  SAVED  TRAINING  COST  SAVINGS 

hundreds  of  man  years  million*  of  dollar* 


500  1000  1500  2000 

ANNUAL  COURSE  DEVELOPMENT  COST  (C  ) 
dollars  per  course  hour 
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FIGURE  7  EFFECTS  ON  DOLLAR  AND  MANPOWER  SAVINGS  FOR  SAMPLED  U  S.  AIR 
FORCE  TRAINING  COURSES  FROM  VARYING  I  T.  DELIVERY  COST  (C  ) 
AND  COURSE  DEVELOPMENT  COST  (C  )  FOR  CONSTANT  ft(CT) 


TRAINING  TIME  SAVINGS  AS  PERCENT  OOLLAR  SAVINGS  AS  PERCENT 

Of  total  training  time  of  total  training  cost 


TRAINING  COST  SAVINGS 


r 


FIGURE 
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EFFECTS  ON  DOLLAR  AND  MANPOWER  SAVINGS  FOR  SAMPLED  U.S  NAVY 
TRAINING  COURSES  FROM  VARYING  I.T.  DELIVERY  COST  (C  )  AND  COURSE 
DEVELOPMENT  COST  (Cc>  FOR  CONSTANT  5<CT>  J 
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A  systematic  shift  in  the  values  of  6(CT) ,  which  can  be  con¬ 
sidered  a  major  variation  in  learning  strategy,  produces  considerable 
variation  in  the  behavior  of  the  savings  curves.  Figures  3,  4,  and  5 
display  these  effects  on  savings  as  6(CT)  is  systematically  decremented 
first  by  one  and  then  by  two  categories,  referred  to  symbolically  as  6^ 
and  6^,  respectively.  The  6^  value  represents  the  best  estimate  of  all 
the  6(CT)  's.  Under  6^  conditions,  all  courses  in  Category  V  would  be 
reduced  to  Category  IV;  Category  III  courses  would  be  classified  as 
Category  II,  and  so  on.  In  the  6q  curves,  all  courses  that  were  Category 
V  would  be  reduced  to  Category  III;  courses  in  Category  IV  to  Category  II, 
and  so  on.  The  solid-line  curves  show  the  results  based  upon  estimated 
values  of  6(CT)  and  fixed  hardware  costs  of  $0.50,  $1.00,  $2.00,  and 
$4,00  per  terminal  hour.  The  dashed  and  dotted  curves  reflect  savings 
based  upon  the  decremented  values  of  6(CT)  and  the  same  hardware  cost 
ranges . 

The  variations  in  6(CT)  yield  tremendous  variations  in  savings. 
As  an  example,  consider  the  case  of  the  Air  Force  sample  (Figure  4  ) . 

For  a  hardware  cost  of  $0.50  per  terminal  hour  and  a  total  courseware 
cost  of  $2500  per  course  hour,  the  savings  increase  from  $2.5  million  to 
$14.4  million  when  6^  is  changed  to  &0.  This  is  an  increase  of  more 
than  500  percent.  This  reflects  the  considerable  impact  of  learning 
strategies — the  major  components  of  5(CT) — on  savings. 

The  estimated  annual  dollar  and  manpower  savings  for  the  sample 
sets  of  courses  selected  from  each  military  service  are  summarized  in 
dollars,  in  training  time  saved,  and  in  percentages  of  the  total  manpower 
and  the  total  training  cost.  The  results,  taken  from  the  computer  runs, 
are  shown  graphically  in  Figures  6,  7,  and  8.  For  example,  the  upper 
graph  in  Figure  6  shows  total  annual  dollar  savings  for  the  39  sampled 
Army  courses.  Each  curve  represents  a  different  assumption  for  hardware 
cost.  The  abscissas  common  to  both  sets  of  curves  show  annual  course 
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development  cost  (C^)  .  Savings  in  millions  of  dollars  and  in  hundreds 
of  man-years  can  be  read  directly  from  the  left  hand  vertical  axis.  On 
the  right  hand  vertical  axis  the  savings  may  be  read  directly  in  terms 
of  percent  of  training  time  or  cost  saved. 

Assuming  hardware  costs  of  $1.00  per  terminal  hour  and  $2500 
total  cost  per  course  development  hour,  the  savings  amount  to  $13.6  million, 
or  18  percent  of  present  costs,  for  the  sampled  Army  courses.  There  are, 
of  course,  some  additional  questions,  not  addressed  in  this  study,  as  to 
how  these  savings  would  be  actually  realized. 

To  realize  these  savings,  it  would  be  necessary  for  the  services 
to  institute  immediate  reassignment  of  graduated  personnel  to  productive 
jobs.  There  is  also  the  possibility  of  giving  such  personnel  additional 
training.  Such  questions  impact  on  military  policy  and  cannot  bo  dis¬ 
cussed  here.  Thus,  these  estimated  savings  should  be  viewed  as  potential 
increases  in  "effectiveness.'' 

In  computing  the  totals  shown  by  the  curves,  savings  for  each 
course  that  yielded  negative  savings  through  the  introduction  of  I.T. 
were  set  to  zero.  Negative  savings  imply  that  the  cost  of  implementing 
I.T.  exceeds  the  projected  savings,  end  there  would  be  no  rationale  for 
making  such  changes. 

The  following  considerations  determined  the  range  of  hardware 
costs.  A  PDP-11,  NOVA,  or  Hewlett-Packard  minicomputer  system  with  20 

Teletype  terminals  costs  approximately  $50,000.  Amortization  over 

* 

five  years  and  use  of  2000  hours  per  terminal  yields  a  cost  of  $0.25 

* 

Both  hardware  and  courseware  costs  were  amortized  over  five  years  to 
an  assumed  salvage  value  for  equipment  and  courseware  of  zero;  i.e., 
it  is  assumed  that  both  hardware  and  courseware  must  be  completely 
replaced  every  five  years. 
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pet  terminal  hour ,  Assuming  maintenance  costs  of  one  percent  per  month 
and  capital  costs  of  5  percent,  the  present  cost  per  terminal  hour 
approaches  $0,50.  In  comparison,  a  Datapoint  2200  stand-alone  system 
with  alphanumeric  display,  tape  cassettes,  and  8K  (byte)  memory  costs 
$8,000.  Assuming  the  same  usage  and  amortization,  this  system  lias  a 
hardware  cost  of  $0.80  per  terminal  hour.  Generally,  costs  of  large 
scale  systems  and  commercially  available  services  range  frail  $0,40  to 
$1.60  per  terminal  hour,  including  software  and  maintenance.  Public 
schools  in  the  San  Francisco  Bay  Area  are  presently  obtaining  CBI 
services  for  $250  per  month  for  unlimited  usage,  including  maintenance, 
aie  Teletype  rental,  and  local  communications  charges.  This  corresponds 
to  a  cost  of  $1.40  per  terminal  hour.  These  costs  cover  the  range  of 
presently  quoted  prices — or  exceed  those  priccs--for  nearly  all  I.T.-CBI 
systems  under  consideration.  An  extended  range  of  representative  hardware 
costs  per  terminal  hour  wore  used  in  this  study:  0.50,  $1.00,  $2.00,  and 
$4.00.  The  lower  figure  reflects  the  assumption  that  hardware  costs  are 
expected  to  decrease. 

In  addition,  the  following  values  of  total  courseware  costs 
per  hour  were  used  in  the  study  to  determine  their  effect  on  I.T.  savings: 
0,  $2500,  $5000,  $7500,  and  $10,000.  The  corresponding  amortized  annual 
costs  were  0,  $500,  $1000,  $1500,  and  $2000,  respectively.  A  total  cost 
of  $2500  ^course  hour  corresponds  to  200  hours  of  authoring  time  at  the 
rate  of  $12.50  per  hour. 

The  graphs  in  Figures  3  through  8  were  compiled  from  the  larger 
set  of  cou rse-by -course  detailed  results  shown  in  the  printouts  presented 
in  Appendix  C.  The  printouts  display  typical  profiles  of  dollar  and  man¬ 
power  savings,  which  can  he  used  as  standards  of  comparison  for  other 
courses  of  interest. 


61 


Figure  9  is  an  illustrative  example  of  the  printouts  contained 
in  Appendix  C.  The  column  headings  are: 


A  lbreviation 


Explanation 


ip 

Throughput  (graduates 

per  year) 

CT 

Course  time  (weeks) 

$TOT 

Total  cost  (dollars  per  year) 

CAT 

Estimated  category  of 

6(CT) — see  Table  ! 

$COMP 

Cost  of  computerizing 

(equipment) 

$SAV 

Total  dollar  savings 

per  year 

MANSAV 

Total  man-years  saved 

per  year 

REF 

Reference  number  for  identifying  source 
documentation — see  Appendix  C 
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V  MAJOR  FINDINGS 


Tlu>  impact  of  I.T.  on  DoD  activities  can  be  inferred  from  the  results 
presented  in  Section  IV'.  1'he  results  suggest  that  if  innovations  in  I.T. 
are  implemented  in  t lie  military  services,  tin.  DoD  will  benefit  from  re¬ 
duced  total  training  costs.  This  section  discusses  the  major  findings 
of  this  research  program.  Discussion  of  the  findings  responds  directly 
to  the  categories  (1)  and  (2)  of  the  research  objectives  discussed  in 
Section  III.  Recommendations,  category  (3),  are  discussed  in  Section  VI. 

A .  Measurement  of  Benefits 

Present  experimental  da*a  on  training  innovations  relates  to  the  DoD 
training  picture  as  a  whole  through  a  quantifiable  parameter  defined  as 
j(CT).  Because  most  experiments  are  reported  in  terms  of  reductions  in 
time  to  meet  course  objectives,  (CT)  was  selected  as  the  central  parameter 
for  the  calculations.  The  cost  calculations  can  be  considered  as  a  simple 
extension  from  estimated  6(CT)s  to  cost  and  manpower  savings. 

Each  military  service  minimizes  training  costs  within  the  constraints 
of  military  readiness,  availability  of  resources,  and  the  levels  of  com¬ 
petence  required  to  carry  out  explicit  tasks.  Within  budget  and  resources 
limitations,  tradeoffs  are  made  that  give  due  consideration  to  all  of  the 
variables.  When  new  policies  are  adopted,  when  research  programs  arc  un¬ 
dertaken,  or  when  training  innovations  are  made,  the  cost  model  permits 
rapid  reassessment  of  the  implications. 

The  total  cost  variations  shown  in  Figures  3  through  8  have  con¬ 
siderable  sensitivity  to  f (CT)  but  less  sensitivity  to  variations  in 
hardware  and  courseware  costs.  Although  hardware  and  cc  ’sewarc  cost 
variations  have  a  large  relative  effect,  this  effect  is  dominated  by  the 
effect  ot  >(CT)  variations.  For  the  Army  sample,  an  increase  in  the 
total  development  cost  from  $2500  to  $5000  per  course  hour  (holding 
delivery  cost  at  $2,00  per  terminal-hour  and  6(CT)  at  6  )  reduces  the  tost 
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saving  by  21  percent.  A  decrease  in  delivery  cost  from  $2.00  to  $1.00 
per  terminal-hour  (at  constant  $2500  per  course  hour  and  6  )  increases 
the  cost  saving  by  25  percent.  A  decrease  in  6(CT)  by  0.1  from  6^  to 
6^  (at  $2500  per  course  hour  and  $2,00  per  terminal-hour)  decreases  the 
cost  saving  by  1$  percent.  A  change  fr >m  6  to  6^  (at  $2500  per  course 
hour  and  $2.00  per  te rm i na 1 -hour)  decreases  the  cost  saving  by  70  percent 
Similar  tradeoffs  hold  for  t lie  Navy  and  Air  Force  samples.  Thus  changes 
in  6(CT)  have  string  effects  on  cost  and  manpower  savings,  whereas  the 
cost  of  hardware  and  courseware  have  a  lesser  effect. 

The  results  suggest  that  considerable  cost  and  manpower  savings  can 
be  realized  by  introducing  I.T.  and  that  these  savings  are,  to  some  ex¬ 
tent,  independent  of  equipment  and  courseware  costs.  Economies  of  scale 
are  not  necessary  to  achieve  cost  and  manpower  savings.  The  most  influ¬ 
ential  variable  is  training  strategy.  Thus,  proposed  I.T.  systems  should 
be  analyzed  on  Lite  basis  of  their  instructional  methods;  less  emphasis 
should  be  placed  on  their  equipment  or  courseware  costs. 

This  principal  relationship  of  savings  to  6(CT)  depends  on  large  values 
of  6(CT)  that  ield  large  reductions  in  training  time  and  corresponding 
large  reductions  in  expenditures  in  student  salaries  while  in  training. 

Since  salaries  are  major  training  costs,  these  results  are  not  surprising. 

A  quick  scan  of  Figure  9,  fer  example,  demonstrates  that  despite 
variations  in  6(CT)  ,  the  "CAT"  variable,  there  are  courses  in  the  area  of 
technical  training  where  it  is  well  worthwhile  to  introduce  I.T.  In  other 
cases,  the  net  savings  are  negative,  indicating  that  the  cost  of  adopting 
I.T.  exceeds  the  projected  benefits.  Several  factors  may  be  responsible 
for  the  negative  savings:  low  throughput,  low  conventional  training  costs, 
or  the  inappropriateness  of  I.T.  for  that  particular  course. 

There  are  several  reasons  why  I.T.  can  be  expected  to  yield  greater 
benefits  than  those  presented.  This  project  investigated  only  technical 
courses.  Enough  evidence  has  been  accumulated  to  indicate  that  the 
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introduction  of  I.T.  in  nontechnical  courses  would  offer  bene  ills  >i 

similar  magnitude.  Also,  overseas  and  other  commands  were  not  covered, 

and  OJT  courses  wore  excluded.  In  the  cast  ol  tin  Arnij  ,  only  CONAKt 
* 

data  were  used. 

The  Army,  as  well  as  the  Navy,  and  Air  Force,  does  not  explicitly 
account  for  all  training  costs,  in  particular  OAT,  For  example,  after 
basic  training,  the  trainee  in  CON' ARC  takes  either  OJT  or  advanced  tech¬ 
nical  training  to  qualify  in  a  more  advanced  military  occupation" 1  spe¬ 
cialty  (MOS).  The  technical  training  courses  for  the  advanced,  technical 
MOSs  vary  in  length  from  5  to  35  weeks  and  are  given  at  a  number  of  differ¬ 
ent  schools.  Although  advanced  individualized  training,  support  training, 
and  advanced  technical  courses  all  have  formal  schools  co  ns  associated 
with  them,  the  trainee  may  bypass  them  and  gain  his  MOS  through  0.J  i  .  No 
method  presently  exists  to  separate  the  portion  of  a  soldier's  time  that 
involves  learning  a  skill  from  the  portion  spent  qualifying  for  an  MOS 
while  attached  to  an  operational  unit.  With  the  application  of  new  I.T. 
within  the  operational  units  (general  purpose  and  strategic  forces),  poten¬ 
tial  savings  would  bo  gained  no*,  only  from  lormal  training  courses  (those 
for  which  statistical  savings  can  be  calculated),  but  also  from  many  OJT 
courses  as  well.  This  would  increase  any  savings  calculated  on  the  basis 
of  available  cost  statistic*,  which  focus  only  on  the  costs  of  formal 
training  courses  and  thus  underestimate  the  actual  amount  of  training  re¬ 
quired  to  reach  a  given  MOS. 

The  Navy  course  called  "Basic  Electricity  and  Electronics,"  discussed 
in  Section  IV,  is  another  example  of  how  I.T.  benefits  can  be  easily 
underestimated.  The  I.T.  used  in  this  course  is  self-paced  learning, 
and  the  catalog  reflects  a  possible  completion  time  ol  one  to  eight  weeks. 

* 

The  Army  has  recognized  the  usefulness  of  I.T.  in  training  other  than 
that  conducted  in  formal  schools.  It  is  low  experimenting  with  the  use 
of  audiovisual  divices  in  selected  Army  divisions  in  CONUS  and  Europe. 
Courses  are  being  redesigned  along  the  tin  s  described  by  the  Hoard  of 

•)  n 

D\  nam  i  c  Tra  i  ning .  “ 
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The  stated  average  completion  time  was  four  weeks  at  San  Diego  and  six 
weeks  at  Great  Lakes.  The  Navy  has  found  that  the  introduction  of  I .T .  is 
an  iterative  process  in  which  further  reductions  in  course  time  occur  as 
new  I.T.  techniques  arc  implemented.  Thus,  the  course  time  averages  may 
still  have  the  potential  for  further  reduction. 

B.  Commonality 

The  services  already  share  a  portion  of  their  training  activities. 
Some  specialty  courses  taught  by  one  service  are  given  to  trainees  from 
other  branches.  This  is  particularly  true  where  machinery,  such  as  avia¬ 
tion  equipment,  is  common  to  the  services.  In  other  cases,  courses  in 
volving  similar  training  areas  are  taught  separately  because  of  the  diffei 
ent  objectives  of  each  service.  Increased  commonality  would  have  the 
advantages  listed  below. 

•  By  jointly  sponsoring  I.T.  research,  the  services 
could  reduce  their  individual  costs  and  gain  the 
benefits  of  mul ti sponsored  research. 

•  By  sharing  operating  experiences  in  I.T.,  the 
services  could  reduce  the  redundancy  of  potential 
errors  made  in  introducing  new  I.T. 

•  By  pooling  their  training,  the  senices  could  achieve: 

Increased  teacher  proficiency. 

Reductions  in  the  number  of  schools. 

Simplified  development  and  revision  of 
curricula  and  exchange  of  courseware 
modules . 

-  Centralized  logistic  and  scheduling  contiols. 

•  By  working  together  in  common  areas,  the  services  could 
improve  the  standardization  of  languages  and  communica¬ 
tions  schemes. 
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These  commonalities  can  evolve  in  a  practical  way  with  time  and  ex¬ 
perience,  especially  il  the  training  commands  are  aware  oi  the  potential 
benefits  they  would  receive  when  procuring  new  I.T.  equipment  and  sponsor¬ 
ing  new  research  and  development.  If  conflicts  should  arise  from  differing 
training  objectives  and  philosophies,  scheduling  incompatibilities,  varia¬ 
tions  in  curriculum  planning,  and  divergent  equipment  requirements,  training 
managers  could  minimize  them  through  joint  participation  in  the  activities 
and  projects  recommended  in  Section  VI. 

Although  commonality  can  help  to  maximize  the  benefits  from  I.T., 
increased  commonality  is  not  necessary  to  gain  benefits  from  I.T.  Benefits 
can  be  realized  from  I.T.  in  each  service  without  a  pooling  of  resources 
and  activities.  In  fact,  some  advantage  might  be  gained  from  independent 
approaches  to  similar  problems. 


C.  Training  Philosophies 

in  the  DoD  training  system  about  half  of  the  training  occurs  on  the 
job;  the  rest  is  formally  organized  as  schooling.  For  all  training,  however, 
I.T.  characterized  by  training  aids  wherever  the  trainee  is  located  offers 
important  potential  savings.  This  complements  a  policy  trend  toward  more 
training  in  the  field  under  realistic  circumstances  away  from  artificial 
school  environments.  The  full  range  of  didactic  material,  deemed  necessary 
to  attain  course  objectives  can  be  conveyed  through  minicomputers  and  video 
devices  on  site  for  a  given  course.  A  trainee  working  toward  qualification 
for  operating  new  equipment  or  achievement  of  a  higher  rating  can  take  his 
course  in  situ,  supplied  by  course  materials  prepared  at  the  school. 

Potential  benefits  from  implementing  training  philosophies  through  I.T 

are  significant: 

•  Reduced  pay,  allowances,  and  relocation  costs  when 
trainee  transits  to  the  school  are  eliminated. 

•  More  efficient  training  as  a  result  of  using  more  realistic 
training  environments. 
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•  Improved  readiness  of  equipment  due  to  its  systematic 
exercise  through  didactic  modes. 

•  Shorter  learning  times  because  instruction  is  individ¬ 
ualized  and  personalized. 

•  Reductions  in  school  overhead  costs  due  to  reductions  in 
school  loadings. 

•  Easier  revision  of  course  material  to  reflect  changes  in 
policy  or  equipment.  (For  example,  new  courseware  can  be  re¬ 
corded  on  cassettes  at  the  course  center  and  mailed  out  to  those 
training  terminals  where  the  course  is  currently  supported.) 

The  minor  redirection  of  training  philosophy  necessary  to  realize  these 
benefits  is  underway.  A  strong  trend  toward  training  in  the  field  already 
exists,  and  the  idea  is  popular  among  field  commanders  who  believe  that 
training  is  more  effective  when  conducted  under  actual  field  conditions. 

If  the  transition  to  I .T. -implemented  training  is  made  slowly  and  continuously, 
time  and  experience  will  permit  procedures,  policies,  and  equipment  to  be  intro¬ 
duced  in  an  orderly  manner  without  disrupting  normal  operations.  This  will 
allow  sufficient  time  for  feedback  from  the  field  experience  to  make  equipment 
and  courseware  modifications. 
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VI  RECOMMENDATIONS 


The  DoU  should  establish  l.T.  research  centers  or  improve  present 
educational  facilities  to  coordinate  research  activities  and  to  accelerate 
the  application  of  research  results  from  the  laboratory  to  the  field. 

A .  DoD  Research  on  Instructional  Technology 

The  financial  risk  to  .  DoD  of  undertaking  future  research  on  l.T. 
is  small  compared  with  the  potential  gains  in  dollar  and  manpower  savings. 
Flexibility  in  mission  readiness  and  preparedness  has  been  shown  to  be  a 
corollary  benefit  from  I.T.,  and  the  military  should  continue  to  consider 
these  requirements  when  establishing  its  research  budgets.  However,  reli¬ 
able  manpower  and  dollar  savings  alone  justify  further  l.T.  research  programs. 

The  potential  gains  to  the  military  of  the  research  programs  suggested 
below  lie  in  their  commonality  to  military  services,  in  their  research 
nature — as  opposed  to  operational  or  developmental  programs — and  in  their 
ease  of  integration  with  projected  IT.  systems.  Their  purpose  is  to 
guide  and  to  clarify  an  optimum  future  development  path  for  l.T.  within 
the  military. 

Five  domains  of  research  are  pertinent: 

•  Theory  of  instructional  strategies 

•  Curricula  and  courseware  development 

•  Course  evaluation  methods 

•  Hardware  and  software  aspects  of  applied  l.T. 

•  Benefit  and  cost  analysis  of  l.T. 

The  results  of  this  study  indicate  that  future  funding  should  be  assigned 
to  these  categories  of  research  on  a  priority  basis  according  to  the  ranking 
in  the  above  list.  Learning,  for  example,  definitely  lias  a  first-order  effect 
on  5 (CT ) ,  and  it  is  a  function  of  the  instructional  strategies  and  evaluation. 
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The  courseware,  hardware,  and  software  effects  were  shown  in  Section  IV  to 
have  less  influence  on  savings  and  should  therefore  be  assigned  lower 
priorities.  Benefit  and  cost  analysis  research  should  be  continued  to  keep 
the  I.T.  costs  in  line  with  the  projected  and  realized  benefits  and  to  im¬ 
prove  the  chances  of  benefitting  from  future  research  results. 

The  Theory  of  Instructional  Strategies 

Research  on  learning  processes  is  necessary  to  the  successful 
expansion  of  I.T.  The  major  component  of  savings  that  can  be  realized 
through  the  application  of  I.T.  to  training  lies  in  this  category. 

Further  research  is  needed  to  answer  the  following  questions; 

•  What  are  the  appropriate  instructional  strategies? 

•  How  can  I.T.  improve  motivation  and  achieve  the  corresponding 
inherent  benefits  of  increased  learning  and  retention,  reduced 
training  time,  and  more  reenlistments? 

•  What  is  the  effect  of  feedback? 

•  What  is  the  role  of  the  teacher  using  I.T.? 

•  How  does  the  Hawthorne  effect*  impact  on  I.T.? 

•  What  is  the  importance  of  interstudent  discussion? 

•  What  is  the  psychology  of  the  student-machine  interface? 

Research  should  be  conducted  to  improve  understanding  of  the  relationships 
between  instructional  strategies  and  (1)  course  content,  (2)  student  popu¬ 
lation  characteristics,  (3)  instructor  training,  and  (4)  CBI  apprentice¬ 
ship  techniques.  Improved  techniques  are  needed  for  interpreting  student 


The  Hawthorne  effect  refers  to  initial  interest  in  and  good  performance 
as  a  result  of  an  innovation.  A  reversion  to  original  performance  occurs 
when  the  novelty  has  passed. 
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responses,  soliciting  desired  responses,  and  increasing  student  satis- 
f  act  ion  . 

Research  on  the  response  ot  low-aptitude  personnel  to  various  types 
of  self-pacing  and  individualized  training  should  be  accelerated.  In  addition, 
research  should  be  conducted  to  determine  improved  combinations  of  peer 
training,  self-pacing,  hands-on  learning,  and  conventional  lecture  study 
with  books  and  manuals.  Relationships  between  instructional  strategies 
and  cognitive  development,  attitude  changes,  motor  skills,  and  group/team 
performance  should  also  be  studied. 

Research  is  necessary  to  improve  identification  and  measurement 
of  the  relative  effectiveness  of  motivational  factors.  The  role  of  the 
instructor  as  a  motivational  model  should  be  studied,  and  improved  methods 
for  fostering  practical  attitudes  in  instructors  toward  I.T.  innovations 
should  be  developed.  This  may  require  the  establishment  of  courseware 
development  as  a  professional  discipline.  Research  is  needed  on  the 
effective  use  of  available  materials  and  games  to  teach  skills  and  concepts 
and  to  provide  practice  in  applying  concepts.  Motivation  is  closely  linked 
with  student  strategies,  as  the  student's  strategy  varies  in  response  to 
the  instructional  strategy. 

Research  on  the  psychology  of  the  student-machine  interface  is 
recommended  for  the  purposes  of  developing  more  effective  modes  of  display, 
improving  the  mechanical  processes  for  student  responses,  and  establishing 
more  efficient  methods  of  structuring  machine  dialog.  The  study  of  this 
nonverbal  dialog  should  include  research  on  more  efficient  man-machine 
interaction. 

2.  Curricula  and  Courseware  Development 

Guidelines  should  be  developed  for  increasing  commonality  in 
curricula  and  courseware  (languages  and  software).  Some  of  the  main 
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impediments  to  dissemination  of  I.T.  courseware  and  Clil  programs  are  in¬ 
compatibilities  in  author  languages,  hardware,  and  software.  For  example, 
two  bases  may  teach  the  same  course,  but  the  CBI  used  for  that  course  at 
one  base  may  not  be  easily  transferable  to  the  other  base  because  of 
software  differences. 

Courseware  development  should  be  established  as  a  separate 
discipline,  fundamentally  independent  of  course  content  and  service. 
Research  is  needed  on  training  authors  and  instructors  on  a  large  scale. 
Although  courseware  costs  impact  in  a  second-order  sense  on  I.T.,  some 
research  should  be  audressed  toward  the  more  efficient  preparation  of 
courseware,  in  particular,  programming  languages  and  formal  processes  for 
creating  teaching  algorithms.  The  wide  range  of  parameter  values  for 
courseware  costs  used  in  the  cost  analysis  in  Section  IV  are  due  to  widely 
varying  estimates  of  the  cost  of  producing  courseware.  The  lower  figures 
(see  Section  IV)  for  courseware  preparation  costs  are  both  reasonable  and 
conservative,  especially  when  viewed  from  the  perspective  of  future 
advances  in  programming  languages,  such  as  the  development  of  automatic 
courseware  generators. 

Courseware  costs  can  be  expected  to  be  reduced  in  the  future 
lor  several  additional  reasons: 

•  Improved  author  experience  and  establishment  of  a 
courseware  discipline. 

•  Use  of  advanced  students  as  authors  of  course  segments. 

•  Appropriate  modularizations  of  course  materials. 

In  conducting  research  in  this  area,  special  consideration 
should  lie  given  to  a  systematic  exploitation  of  research  results  on 
courseware  organization,  since  this  is  one  area  where  research  results 
are  not  adequately  disseminated. 
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Evaluation 


I.T.j  and  CB1  in  particular,  is  still  at  too  early  a  stage  in 
its  development  to  have  established  accepted  standards  of  evaluation 
Research  is  necessary  to  develop  models  for  comparing  varieties  o!  l.T. 
systems  on  a  common  basis.  Course  evaluation  methods  must  be  determined 
that  meet  the  following  objectives: 

•  To  assure  attainment  of  course  training  objectives 

•  To  meet  go/no-go  testing  criteria 

•  To  provide  feedback  on  the  student's  progress 

•  To  measure  the  trainee's  performance 

•  To  measure  the  trainee's  confidence 

•  To  provide  processes  for  identifying  learning  deficits 
and  for  applying  appropriate  remedial  action. 

Hardware  and  Software 

Research  in  l.T.  hardware  (computer  and  other)  and  software 
should  be  considered,  even  though  its  impact  on  savings  is  less  than  that 
attainable  through  improved  instructional  strategies.  The  term  "hardware 
is  used  here  to  refer  to  the  collective  media  and  equipment,  although 
computing  equipment  has  been  emphasized.  The  principal  objectives  of  any 
research  on  hardware  should  be  first,  to  improve  the  training  mechanism, 
and  second,  to  improve  cost  effectiveness. 

In  the  past,  hardware  and  software  availability  have  often 
determined  the  constraints  of  CBI  courseware.  This  relationship  is  not 
appropriate  for  the  emerging  technology.  With  appropriate  research,  the 
process  can  be  reversed,  and  learning  processes  and  l.T.  requirements  can 
be  used  as  a  basis  for  selecting  and/or  developing  the  appropriate  hard¬ 
ware  and  software.  Such  an  approach  requires  research  in  several  areas: 
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•  The  application  of  special  purpose  centra]  processing  units 
and  peripheral  equipment  in  I.T.  environments. 

•  Relationships  between  training  objectives,  instructional 
strategies,  and  different  hardware  configurations. 

To  benefit  from  research  on  hardware/media  conducted  in  the 
industrial  sector,  additional  economic  research  is  recommended  to  determine 
means  of  fostering  and  accelerating  the  response  of  the  private  indus¬ 
trial  sectors  to  the  requirements  of  I.T.  Justifications— based  on  sound 
business  principles — for  the  development  of  competitive  sources  of  I.T. 
hardware  need  to  be  presented. 

SRI  recommends  that  software  research  concentrate  on  establishing 
criteria  for  the  selection  of  author  languages,  based  on  training  objec¬ 
tives  and  instructional  procedures.  More  research  on  (1)  software  tech¬ 
niques  for  data  collection  and  evaluation  in  real  time  and  (2)  basic 
software  components--e . g . ,  metacompilers  and  translators--would  also  be 
useful . 

In  particular,  note  that  the  number  of  CI11  languages  can  only 
proliferate  with  increasing  time.  Thus,  emphasis  must  be  placed  on  de¬ 
veloping  metacompiler  generators  for  languages  that  concentrate  on  the 
language  aspects  rather  than  the  software  considerations. 

5.  benefit  and  Cost  Analyses 

Most  of  the  benefit  appraisals  of  I.T.  conducted  in  the  last 
faw  years  have  been  evaluated  using  cost  as  the  principal  parameter  of 
interest  (see  Appendix  A).  In  each  of  these  cases,  appraisals  were 
complicated  by  a  lack  of  comparable  bases  for  cost  comparisons.  Research 
is  required  for  the  development  of  a  uniform  approach  in  which  cost  variables 
can  be  fixed  and  varied  systematically  to  provide  a  controlled  context  for 
evaluation. 
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Tickton  and  Kotin  delineated  many  of  the  criteria  needed  ior  such 
20,28 

an  analysis.  If  their  plans  are  adopted  by  national  agencies,  then 

all  grants  rom  such  agencies  will  require  the  grantee  to  provide  the 
specified  information  needed  by  the  costing  methods.  Such  a  procedure, 
if  also  adopted  by  the  military,  would  enable  the  military  to  test  its 
programs  against  a  standard  and  to  compare  its  programs  with  programs  used 
in  the  civilian  and  industrial  sectors.  Sit  I  recommends  additional  research 
in  this  area  to  facilitate  the  use  of  such  costing  methods  in  the  military. 

The  experience  gained  during  this  research  project  demonstrated  that  the 
absence  of  standardized  costing  procedures  imposes  a  handicap  on  researchers 
studying  I.T.  in  the  military.  The  recommended  research  is  intended  to 
eliminate  this  handicap. 

A  systematic,  course-by-course  analysis  based  on  the  data  base 
and  an  automated  mathematical  model  could  be  useful  to  each  of  the  services. 

However,  such  research  should  incorporate  all  up-to-date  information  on  courses 
that  have  already  been  modified  for  I.T.  (e.g.,  programmed  instruction). 

Research  on  data  base  methodology  for  CBI  systems  is  recommended. 

Considerable  disagreement  exists  about  the  appropriate  data  requirements 
for  CBI  systems  (in  such  areas  as  data  logging  and  the  size  of  the  data  base). 

Research  should  be  conducted  to  determine  the  necessary  data  base  requirements. 

Assumptions  that  large  data  bases  are  necessary  should  be  studied  carefully, 
since  there  is  no  generally  accepted  evidence  that  sizeable  portions  of  CBI 
data  bases  are  useful.  The  result  of  such  assumptions  may  turn  out  to  be 
large,  costly,  inflexible  systems. 

Software  costs  can  easily  dominate  the  total  computing  system  costs 
of  CBI.  Thus,  research  on  computing  software  should  have  the  same  priority 
level  as  the  hardware  research  in  CBI  systems. 

A  systematic  analysis  would  also  aid  in  t  lie  development  of  an 
analytical  tool.  The  main  purpose  ol  perfecting  an  analytical  tool 
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is  to  cnaolc  a  paraprofessional  10  use  it  in  evaluating  courses.  At 
present,  this  is  strict  1\  the  .job  ol  a  highly  skilled  professional. 

Titus,  SRI  recommends  additional  research  to  continue  the  study  ol  t  lie 
relationship  between  MCI)  and  the  independent  variables  ol  I.T.  Such 
information  would  be  especially  useful  in  planning  curriculum  modifica¬ 
tion  and  development  in  operational  I.T.  research  centers.  It  would 
provide  a  systematic  basis  for  relating  the  capabilities  of  the  instruc¬ 
tional  techniques  to  the  instructor's  detailed  knowledge  of  course  con¬ 
tent  and  objectives.  Such  research  should  be  directed  toward  investigating 
functional  relationships  between  (CT)  and  the  variables,  determining 
domains  of  values  for  the  variables,  and  validating  the  functional  form 
using  the  existing  data  base. 

B.  Instruction  Technology  Research  Centers 

To  improve  coordination  of  activities,  expenditures,  and  research 
among  the  services,  it  is  necessary  to  establish  a  basic  I.T.  research 
center  and  to  enhance  the  present  operational  I.T.  eflorts  within  the 
services.  Both  classes  of  I.T.  centers  are  important  because  the  services 
are  obligated  under  mission  requirements  to  concentrate  their  principal 
resources  on  operational  programs,  and  because  it  is  necessary  in  this 
long  lead  time  technology  to  support  basic  research  via  another  mechanism. 
The  high  commonality  of  this  technology  strongly  argues  for  an  independent 
activity  capable  of  supporting  all  three  services. 

The  concept  of  the  I.T.  research  center  (either  basic  or  operational) 
has  been  presented  by  several  other  researchers.  For  example,  a  compre¬ 
hensive  design  was  completed  in  1967  by  the  Office  of  Naval  Research. 

The  Navy  design  sets  forth  a  rationale  similar  to  the  one  presented  in 
this  report.  However,  due  to  the  evolution  of  technology  in  the  last 
five  years,  the  procedures  recommended  here  are  somewhat  different. 
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The  Basic  Instructional  Technology  Research  Center 


1 . 

The  I.T.  research  center  should  be  structured  as  an  autonomous 
organization  with  well-established  lines  of  communication  to  the  services. 

It  should  be  administered  contractually  under  either  AUPA  or  joint  ser¬ 
vices  management.  The  center  should  be  organized  with  balanced  participa- 
ti  on  ol  all  three  services  in  mind.  Imbalance  in  favor  of  any  particular 
service  would  only  interfere  and  degrade  the  cooperation  that  is  a  neces¬ 
sary  ingredient  to  the  success  of  the  center's  program. 

The  objectives  of  the  basic  I.T.  research  center  should  be  the 

following : 

•  To  provide  a  foundation  for  nourishing  basic  research  in 

I.T. 

•  To  monitor  and  validate  the  future  benefits  of  I.T.  in  the 
DoD. 

•  To  maintain  an  optimum  path  for  the  expansion  of  the  use 
of  i.T.  in  the  Dot). 

The  center  must  perform  and  support  research  to  be  cognizant  of  the  present 
and  projected  state  of  the  art  in  I.T.,  and  it  must  also  take  responsibility 
for  research  in  "blue-sky"  areas  that  the  individual  services  would  have 
difficulty  justifying.  The  center  should  also  be  available  as  a  bread¬ 
board  facility  for  outside  researchers  to  use  in  conducting  their  experi¬ 
ments. 

The  center  should  serve  as  a  focal  point  for  centralizing  docu¬ 
mentation,  communications,  and  research  efforts.  For  example,  many 
laboratory  research  results  on  learning  processes  are  rarely  applied  at  the 
actual  training  site,  or  they  are  only  partially  implemented.  The  center 
could  strengthen  the  feedback  loop  between  research  sites  and  field  sites, 
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and  it  could  assure  some  continuity  in  the  filtering  process  between  the 
laboratory  and  the  classroom  by  monitoring  the  technology. 


The  proposed  center  can  be  similar  to  present  coordinating 
centers  operated  by  the  Office  of  Education,  Atomic  Energy  Commission, 
National  Institutes  of  Health,  and  National  Bureau  of  Standards.  However, 
the  activities  of  the  basic  I»T.  research  center  could  be  dispersed  in 
location,  possibly  along  the  nodes  of  the  ARPANET.  Th(  facilities,  for 
the  most  part,  already  exist.  Since  the  center  must  serve  as  a  timely 
distribution  and  communication  center,  the  ARPANET  can  serve  admirably. 

The  center  must  be  responsible  for  establishing  some  level  of  standards 
for  research  and  programs,  without  restricting  present  efforts.  It  must 
accept  alternative  approaches;  in  fact,  it  should  support  them  with  a 
balanced  perspective.  Thus  the  center  should  be  responsible  for  relating 
the  efforts  of  the  individual  services  and,  thereby,  for  preventing 
unnecessary  redundancy. 

The  center  must  be  organized  to  compare  components  and  technique 
in  relation  to  training  effectiveness  and  compatibility  with  equipment. 
Training  strategies  should  bo  optimized  to  make  use  of  variations  in  equip 
ment . 

One 'of  the  most  important  functions  of  the  center  should  be  to 
assure  the  availability  of  competitive  sources  in  industry  for  hardware, 
software,  and  courseware.  The  industrial  sectors  of  the  economy  are 
usually  very  responsive  to  DoD  technical  requirements,  and  some  central¬ 
ization  of  specifications  for  this  type  of  technology  would  aid  the  cost 
effectiveness  of  industry's  research  and  development  in  this  technology. 

The  ARPANET  and  the  Network  Information  Center  could  be  used 
as  a  means  for  communication  and  as  a  research  facility  to  coordinate  I.T, 
and  CBI  research  funded  by  the  basic  I.T.  research  center.  The  network 


would  permit,  for  example,  experimental  techniques  in  one  laboratory  to 
be  linked  to  another.  Or,  courseware  created  at  one  node  could  be  exercised 
at  another  node  to  be  evaluated  on  different  equipment  and  in  line  with 
varying  objectives. 

The  I.T.  research  center  could  also  serve  an  invaluable  role 
by  actiig  as  a  catalyst  in  instituting  workshop  concepts  and  providing 
workshop  facilities.  There  has  been  considerable  interest29  in  establish¬ 
ing  I.T.  workshops  where  participation  could  be  dispersed  and  easily 
available  to  many  researchers.  Again,  the  ARPANET  can  serve  as  a  valuable 
adjunct  to  this  concept. 

Workshops  provide  an  efficient  process  for  developing  opinions 
and  coordinating  information.  Assertions  can  be  proposed,  discussed,  and 
modified  in  a  matter  of  minutes  or  hours,  rather  than  in  a  period  of 
months.  In  I.T.,  a  number  of  research  activities  are  carried  on  all  over 
the  country.  These  researchers  could  convene  on  tin  ARPANET,  say,  once 
a  month.  Geographically  they  would  be  at  home,  using  their  own  computers 
and  instructional  programs.  Functionally,  however,  these  programs  would 
be  exercised  by  the  entire  community.  Revisions  could  be  tried  and  dis¬ 
cussed  on  line.  The  ability  to  try  alternatives  in  a  group  and  to  iterate 
toward  new  concepts  in  a  short  time  adds  a  qualitatively  different  re¬ 
search  dimension. 

To  fulfill  the  objectives  proposed  above,  the  research  center 
must  have  personnel  skilled  in  all  ol  the  technical  and  administrative 
categories  that  relate  to  the  use  of  I.T.  This  includes  but  is  not 
limited  to: 

•  Mathematicians  and  statisticians 

•  Psychologists 
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•  Economists 


•  Courseware  authors 

•  Software  experts 

•  Hardware  expert  3 

•  Educators 

•  Administrators. 

Such  technical  and  administrative  skills  are  necessary  both  to  perform  and 
to  monitor  research  programs  and  to  keep  the  center  headed  toward  its 
principal  objective:  optimizing  the  benefits  that  can  be  achieved  by  the 
introduction  of  advanced  technology  in  the  military  training  establishment. 

2 .  The  Operational  I.T.  Research  Centers 

Although  basic  research  in  I.T.  should  be  supported  or  coordi¬ 
nated  through  a  joint  service  organization,  the  individual  services  will 
use  the  results  of  such  research  to  meet  their  individual  objectives. 

Tie  services  already  have  established  I.T.  research  facilities  and  the 
suggestions  below  are  intended  to  augment  the  function  of  these  existing 

facilities. 

First,  SRI  recommends  that  each  service  designate  one  of  its 
facilities  as  an  I.T.  operational  communication  center  for  interfacing  with 
the  basic  I.T.  research  center.  Although  the  center  located  at  a  Navy 
installation  might  serve  as  the  main  center  for  Naval  courses,  all  bases 
could  be  informed  on  a  day-to-day  basis  of  I.T.  activities  in  the  other 
services.  Part  of  the  communication  role  of  the  operational  centers  would 


*Keesler  AKB;  Chanute  AFB;  Navy  Training  Devices  Center,  Orlando;  Naval 
Training  Research  Institute,  San  Diego;  Fort  Monmouth. 
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be  to  provide  a  variety  of  I.T.  demonstrations  for  those  interested  in 
initiating  or  expanding  the  use  of  I.T.  at  their  bases.  The  availability 
of  a  single  facility  with  information  on  all  I.T,  courses  in  the  service 
would  tacilitate  implementation  of  I.T.  and  remove  the  problem  of  on¬ 
site  course  demonstrations. 

Second,  the  operational  centers  should  be  responsible  for  training 
instructors  in  the  use  of  I.T.,  particularly  CDI .  Instructors  might  well 
be  trained  at  these  centers  to  return  to  their  bases  to  instruct  others  in 
the  new  role  of  CBI  monitor/instructor .  Feedback  to  the  centers  from  in¬ 
structors  in  the  field  could  optimize  the  training  of  instructors  who  will 
be  using  CBI  in  such  diverse  ways  as  drill  and  practice,  tutorial,  and 
simulation  modes. 

Third,  the  centers  should  train  instructors  to  prepare  I.T. 
curricula  in  a  variety  of  modes.  In  the  course  of  such  training,  some 
courses  will  be  produced  at  the  centers,  but  in  the  long  run,  it  is  assumed 
that  most  courses  will  evolve  in  a  decentralized  fashion  from  local  in¬ 
structors  . 

Fourth,  the  centers  should  disseminate  courses  written  at  these 
centers  and  at  the  basic  I.T,  research  center.  The  operational  centers 
also  oner  an  ideal  location  to  collect  information  on  I.T.  courses. 

FeedLack  from  the  users  of  such  courses  could  be  used  in  revising  the 
material,  as  well  as  in  developing  techniques  for  revision. 

Fifth,  the  centers  should  serve  as  a  vital  link  in  reformulating 
basic  research  results  to  be  of  use  in  the  field. 

Interservice  staffing  should  provide  ongoing  communication  and 
stimulation  to  I.T.  users  producers.  Tim*  stall  ol  each  center  should 
include  exports  on  hardware,  software,  and  courseware.  Courseware  experts 
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should  be  familiar  with  the  preparation  of  a  variety  of  I.T.  modes  and 
with  research  in  learning  applicable  to  I.T.  Software  and  hardware  experts 
should  be  familiar  not  only  with  equipment  at  the  laboratory,  but  also  with 
equipment  likely  to  be  found  at  bases  using  the  laboratory.  Service  person 
nel  using  the  facilities  of  the  laboratory  should  be  experts  in  the  area 
for  which  they  will  be  producing  curricula. 

The  present  study  covered  the  role  of  I.T.  in  the  military 
services  as  a  whole.  Since  existing  military  educational  centers  were 
not  exhaustively  studied,  no  recommendations  are  made  concerning  experi¬ 
ments  to  be  conducted  in  these  existing  centers,  scales  of  operation, 
research  investments,  staffings,  or  other  considerations.  Such  topics 
would  benefit  from  further  study. 
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SOURCES  OF  TRAINING  COSTS  AND  THE  NUMBER  OF  TRAINEES 

This  appendix  presents  descriptions  and  examples  of  t  lie  source 
documents  from  which  the  data  used  lor  this  study  were  collected. 

1  .  U.S.  Ail1  Force 

The  U.S.  Air  Force  compiles  summary  cost  data  on  training  and  main¬ 
tains  them  in  a  computerized  data  bank.  This  data  bank  is  updated  each 
year  and  was  designed  by  the  Management  Analysis  Office  of  the  Financial 
Division  Comptroller's  Office  at  the  Headquarters  of  the  Air  Training 
Command.  Data  arc  compiled  on  total  cost  per  graduate  at  the  individual 
training  course  level.  Cost  data  used  in  this  study  are  based  upon  Air 
Force  experience  in  fiscal  1972. 

Another  source  document  necessary  to  this  study  is  the  programmed 
technical  training  plan  of  the  U.S.  Air  Force.  It  is  used  to  obtain  the 
number  of  programmed  students  to  graduate  from  each  course  in  1972. 

Both  the  cost  and  the  training  plan  data  were  compiled  by  the  Headquarters 
of  the  Air  Training  Command,  located  at  Randolph  Air  Force  Base,  Texas. 

A  multipage  computer  printout  maintained  at  ATC  headquarters  shows 
the  cost  breakdown,  ns  well  as  the  total  cost  per  trainee  graduated, 
adjusted  for  the  attrition  rate.  This  information  was  made  available, 
on  a  course-bv-course  basis,  for  ail  students  involved  in  technical 
training  command  courses  tor  fiscal  1972,  Courses  are  identified  by 
course  number  and  title  in  this  data  bank.  The  cost  definitions  used 
for  the  various  accounts,  as  defined  by  the  comptroller  in  the  Manage¬ 
ment  Analysis  Office  of  the  Financial  Division,  are  given  below. 

•  Direct  Cost s--Those  costs  directly  attributable  to  training 
i.e.,  pay  and  allowances  of  nil  personnel  in  the  school, 
contracted  and  purchased  services  (such  ns  maintenance  and 
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repair  of  training  equipment  anti  Instruction),  supplies 
and  training  aids,  maintenance,  POL,  and  other  miscellaneous 
cost  s. 

•  Support  Costs — The  costs  Tor  those  activities  normally 
considered  to  be  base  support  and  not  directly  engaged  in 
training;  Le.,  military  and  civilian  pay,  maintenance, 
supplies,  materials,  utilities,  rent,  commercial  trans¬ 
portation  and  communications,  printing,  reproduction, 
and  others. 

•  Student  Pay — This  cost  is  computed  using  a  standard  rate 
for  each  rank;  e.g.,  a  captain's  pay  is  based  on  an  average 
of  all  Air  Force  captains’  pay  (including  pay,  allowances, 
and  benefits). 

•  Command  Overhead — This  cost  includes  the  operating  costs 
of  the  command  headquarters  (i.e.,  ATC)  and  also  that 
portion  of  the  support  cosl s  of  the  host  base  (i.e., 

Randolph)  allocated  to  the  headquarters. 

2 .  U.S.  Army 

Within  the  Continental  Army  Command  (CONARC) ,  the  U.S.  Army  has 
a  standard  methodology  to  analyze  the  cost  of  training  for  all  schools. 

It  is  the  basis  for  providing  the  Comptroller  of  the  Army  with  detailed 
cost  analyses  and  costs  per  graduate  for  each  course  taught  at  CONARC 
schools. 

Using  these  raw  cost  data  (fiscal  1971  costs  per  graduated  tiainec) 
collected  by  CONARC,  inferences  were  drawn  about  Army  cost  and  manpower 
savings  resulting  from  the  introduction  of  new  I.T.  Forms  used  by  the 
Army  Training  Command  to  report  these  costs  for  each  course  identify  the 


See  the  annex  to  this  appendix. 
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specific  course  by  title  and  number.  Raw  data  were  tak< n  from  n  sample 
of  the  source  documents  and  then  keypunched  and  stored  for  use  in 
the  analysis. 


Five  fields  of  data  wore  used:  course  title,  course  number,  course 
length  in  weeks,  throughput  in  terms  ol  normal  1  zed  graduates,  and  total 
cost  per  normalized  graduate.  The  course  description  corresponding  to 
the  given  course  number  is  obtained  from  the  Army  School s  Catalog. 

3 .  U.S.  Navy 

In  1971  t  lie  U.S.  Navy  had  13  separate  commands,  each  conducting 
training  courses  for  its  particular  needs.  Each  of  the  13  commands 
maintained  its  own  schools  and  published  its  own  course  catalogs  for 
the  training  offered.  The  13  commands  were: 

(1)  A — Bureau  of  Naval  Personnel 

(2)  B--Bureau  of  Medicine  and  Surgery 

(3)  C — Naval  Air  Training  Command 

(1)  b — Naval  Air  Force,  Atlantic 

(3)  E — Naval  Air  Force,  Pacific 

(6)  F — Submarine  Force,  Atlantic 

(7)  (/--Amphibious  Force,  Atlantic 

(8)  H--Amphibious  Force,  Pacific 

(9)  J--Training  Comma  id,  Atlantic 

(10)  K--Training  Command,  Pacific 

(11)  L--Subma ri no  Force,  Pacific 

(12)  M — Naval  Air  Systems  Command  Representative,  Atlantic 

(13)  N--Naval  Air  Systems  Command  Representative,  Pacific. 


The  upper  case  letter  designator  is  the  first  element  of  a  training 
course  alphanumeric  descriptor  that  identifies  the  sponsoring  command 
for  the  particular  course. 


In  1072  the  Office  of  the  Chief  ol  Naval  Training  began  to  consol i- 
,lato  ,XM>orts  nn  Ns,val  "-iiuiinn  data.  Training  within  t  ho  13  commands 
WMS  rt>ur<>u,K‘(l  in,°  ;l  niiiiiboi-  of  sultconimamls,  including  the  Chief 

of  Naval  Technical  Training  and  the  Chief  of  Naval  Aviation  Training. 

It  is  estimated  that  there  are  some  (it)t)()  training  courses  given  in 
the  T.S.  Navy  and  that  under  the  new  *  raining  command  organization,  some 
5t)  percent  of  these  are  to  le  given  by  the  Naval  Technical  Training 
Command.  They  include  most  of  the  basic  skills  courses;  the  courses 
wtth  the  V  des  ignat  e>r,  originated  In  the  bureau  of  Medicine  and  Surgery 
are  the  exception. 

For  this  report,  the  information  on  training  time,  cost,  number  of 
trainees,  and  related  course  descript  iot  s  is  data  derived  from  several 
Navv  sources .  All  of  the  courses  analyzed  in  this  report  for  potential 
application  of  l.T.  have  either  an  "a"  or  V  preceding  the  numerical 
digits  of  the  course  number.  The.r  course  descriptions  wore  taken  from 
school  catalogs  published  by  the  bureau  of  Naval  Personnel  and  t he  Naval 
Air  Training  Command. 

lie  data  on  trainees  graduated  from  the  courses  were  taken  from 
the  Naval  technical  Training  Command  computer  printout  headed  "basic 
hovel  Training  Data."  For  example,  lor  the  basic  technical  training 
course  tor  electronic  technicians  for  Course  A-l 00-0012,  t he  same 
course  curricula,  content,  and  length  are  given  at  the'  Service  Schools 
Commands  at  Croat  hakes  and  San  Diego.  At  San  Diego  the  course  identifiei 
is  A-l DD- DO  lb.  For  th»  purposes  of  the  present  study,  Courses  A-l 00-001 2 
and  A- 1 0.1-0013  are  treated  as  separate'  elata  entries.  The  actual  number 
<>f  trainees  graduat.-el  from  both  courses  is  taken  from  the  first  quarterly 
re-port  s  of  fiscal  year  1972-1973.  This  number  is  multiplied  by  four  to 
obtain  the  « e>t  r»  I  year’s  e-stlmate'el  throughput  lor  the  t  ue>  eourses 


102 


These  nrintoiils,  made  available  by  the  Naval  Technical  Training 
Command,  are  summaries  of  information  that  originates  at  the  classroom 
level.  Figure  10  is  an  example  of  the  source  document  from  which  these 
summarized  data  are  compiled.  This  form  is  the  basic  report  prepared 
at  the  school  level. 

From  the  Bureau  of  Naval  Personnel  Formal  Schools  Catalog,  the 
course  descriptions  of  these  same  two  con rses-~A- 1 00-001 2  and  A- 1 00-001 S — 
given  at  two  locations  are  obtained.  For  courses  with  a  "c"  designation, 
an  example  would  be  C-000-20I2,  which  was  selected  from  the  school 
catalog  of  the  Naval  Air  Training  Command.  It  is  an  example  of  a  short 
course,  (3  weeks)  that  has  a  high  throughput.  The  estimated  number  of 
fiscal  1972-1973  graduates  is  calculated  as  four  times  the  cumulative 
graduates  shown  for  the  first  quarter  (available  data  at  this  time),  or 
about  6500. 

To  assign  costs  for  the  above  "A'  and  ”c"  courses,  t  lie  researcher 
must  refer  to  another  Navy  publication,  since  costs  arc  not  typically 
reported  and  summarized  on  a  course-by-course  basis  in  the  various 
commands  where  training  is  given.  However,  an  annual  cost  study  per¬ 
formed  by  the  Bureau  of  Naval  Personnel  makes  possible  the  assignment 
of  some  selected  raw  cost  data  on  a  course-hy-course  basis.  As  an 
example  of  the  procedure  used  for  the  Navy  data  under  the  ET-1501  rating, 
a  13-week  course  in  the  ET-A  school  ^r  the  E-2  phase  shows  a  total  cost 
per  student  (including  school,  military  pay,  and  leave  costs)  of  $1,588, 
or  $1,588  13  =  $122  per  week.  This  cost  per  week  is  multiplied  by  the 
course  length,  in  weeks,  to  yield  the  estimated  training  cost  for 
the  course. 

The  second  sample  course  is  entitled  "Aviation  Mechanical  Funda¬ 
mentals  Course,  C-000-2012.  During  the  first  cpiarter  of  1972,  1626 
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trainees  graduated.  Some  0500  arc  expected  to  graduate  if  the  first 
quarter  rate  is  maintained  for  a  year.  The  per  week  cost  for  the 
AMFU  course  assignor  to  these  0500  trainees  for  the  purpose  of  an 
annual  estimated  cost  is  $d29/3,  or  $115  per  week. 
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Annex  to  Appendix  It 


STANDARD  MKTIIODOLOQ  l  SKI)  TO  Dl  TM!M  INI'  Till  COST  I’l.R  CICMH  ATI 
IOR  AI.I,  CONARC  SCHOOLS  AND  ARMY  THAI  NINO  CENTERS 


1.  Purpose. 

a.  To  provide  CONARC  School  Commandants  with  detailed  cost  analyses  and 
costs  per  graduate  for  each  course  taught  in  their  respective  schools. 

b.  To  provide  ATC,  installation  and  CONU  >A  commanders  with  detailed  cost 
analyses  and  costs  per  graduate  for  each  ATP  conducted  at  USATC's  under  their 
command . 


c.  To  furnish  the  Comptroller  of  the  Army  and  DCSPER,  DA  costs  relative 
to  individual  training  conducted  throughout  CONARC. 

2.  Data  Sources. 


a.  Direct  0>j\  and  MPA  cost  data,  ammunition  expenditures,  student/trainee 
nput-output  data,  attrition.  POI  hou’-s  per  course  and  other  pertinent  data 

re  ated  to  the  various  raining  activities  is  furnished  this  headquarters  by 
t  c  rt »pccti\ a  .Schools  and  I'SATC's  in  accordance  with  instructions  contained 
in  letter,  this  headquarters,  21  July  1971,  subject:  CONARC  Cost  Analysis 
Requirements  (KCS  ATCU-l- 159) . 

b.  Expense,  workload,  workforce,  population  and  other  data  used  ir.  the 
eve  opment  of  indirect  (base  operations  and  support)  costs  is  furnished  to 

this  headquarters  by  the  installations  upon  which  training  activities  are 

mMAtCd  1U  arrordancc  witb  letter,  this  headmiar  D.rS ,  21  July  1971,  subject: 
CONARC  Cost  Analysis  Requirements  (RCS  ATCCM-159). 

c.  Installation  and  activity  mechanised  Prior  Year  Reports. 

3.  Methods  of  Computing  Costs  per  Graduate. 

a.  Computation  of  direct  costs. 


(1)  Student  pay  and  allowance 
modal  grade  of  student  (AR  37-29 ) 
A1C  co 5. ts  consideration  must  be  gi 
start  of  training  and  time  between 


s.  Course  length  in  weeks  X  weekly  rate  for 
=  per  graduate  cost.  NOTE:  In  computing 
ven  to  tine  spent  by  trainee  in  awaiting 
completion  of  training  and  shipment. 


(2)  Travel  pay  to  course.  Average  one-way  mileage  (determined  from 
samp  e  of  student  records)  X  $.06  =  travel  pay  per  graduate. 


N  TL .  Tnvcl  .ion  course  is  picked  uo  in  costing  of  units  or  next  place  of 
recruit  training. 
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(3)  Per  diem  at  course  (TOY  courses  only). 

(a)  Officer,  Course  length  in  weeks  X  $35.00  =  cost  per  graduate  (if 
quarters  are  available,  if  not,  per  diem  rate  must  be  obtained). 

(b)  EH.  Course  length  in  weeks  X  $14.00  =  cost  per  graduate. 

(4)  Instructional  departmental  costs  and  ATC  direct  instructional  costs. 

(a)  MPA  -  Compute  total  military  pay  cost  of  each  Instructional  Pept, 
including  instructors,  departmental  overhead,  command  control,  etc. 

NOTE):  Information  is  furnished  in  school/ATC  reports  in  the  form  of  yearly 
military  pay  costs  or  by  average  assigned  strength  by  grade  for  the  FY. 

1  Compute  number  of  classes  for  each  course  taught.  Number  of  classes  = 
classes  started  in  FY  +  %  classes  completed  in  FY. 

2  Compute  total  POI  hours  per  course.  Total  POI  hours  =  POI  hours  per 
course  X  number  of  classes. 

—  Compute  POI  hours  per  graduate.  POI  hours  per  graduate  =  total  POI 
hours  ;  number  of  normalized  graduates.  Normalized  graduates  per  course  = 
actua!  graduates  -  h  in  training  at  beginning  of  FY  +  ^  in  training  at  end 


4  Complete  steps  (_1 )  through  (3)  for  each  course. 

5  Add  total  POI  hours  per  course  for  each  ins trrit tonal  department  that 
expends  any  etfort  into  a  course. 


6  Divide  the  total  departmental  MPA  cost  by  the  total  POI  hours  of  effort 
expended  by  that  department  to  obtain  the  MPA  cost  per  POI  hour.  Multiply  the 
M  cost  pel  POI  hour  times  the  POI  hours  per  graduate  to  obtain  departmental 
cost  per  graduate  for  each  department.  Add  the  per  graduate  cost  for  each 
department  to  obtain  total  MPA  instructional  department  cost  per  graduate. 


(b)  Repeat  the  same  process  to  obtain 
per  graduate  in  categories  of  civilian  pay. 
OMA. 


the  instructional  department  cost 
supplies  and  equipment  and  other 


(5)  School  overhead  and  school  support  command  or  brigade. 

(a)  Determine  total  school  overhead  MPA,  civilian  pay,  supplies  and 
equipment  and  other.  Includes  Commandant,  Office  of  the  Secretary,  DOI , 
supply  and  maintenance  elements,  administrative  support,  ETV,  etc. 

(b)  Determine  total  manweeks  of  training  per  course  and  manweeks  per 
graduate  per  course. 

Total  manweeks  training  per  course  =  (normalized  graduates  +  ^  attritions) 

X  course  length  in  weeks. 


110 


Total  manweeks  per  graduate  =  total  training  manweeks  per  course  f 

normalized  graduates  per  course. 

(c)  Divide  each  cost  element  for  total  school  overhead  by  the  grand 
total  of  training  manweeks  for  the  entire  school  r  arrive  at  a  school 
overhead  cost  per  training  manweek. 

(d)  Multiply  the  results  times  the  manweeks  per  graduate  for  each  course, 
and  divide  by  the  number  of  normalized  graduates  in  each  course  to  arrive  at 

a  cost  per  graduate  by  cost  clement. 

(e)  Repeat  the  same  process  for  the  cost  elements  for  the  school  support 
command  or  school  brigade. 

(6)  Ammunition  -  Total  cost  of  ammunition  expended  per  course  t  normalized 
graduates  per  course  =  ammunition  cost  per  graduate. 

(7)  Major  PEMA  item  depreciation. 

(a)  Using  acquisition  costs  for  all  PEMA  items  issued  to  each  instructional 
dept,  determine  yearly  amortization  costs.  Tanks,  trucks,  artillery  pieces, 
etc.  amortized  over  a  12  year  period;  tactical  radios,  small  arms  and  other’ 
small  items  amortized  in  5  years. 

(b)  Add  yearly  amortization  costs  for  each  dope  to  arrive  at  total  equip¬ 
ment  depreciation  costs  per  dept. 

(c)  Divide  department  depreciation  cost  by  total  department  P01  hours  to 
arrive  at  cost  per  POI  hour. 

(d)  Multiply  the  cost  per  POI  hou  times  the  POI  hours  per  graduate  per 
course  to  arrive  at  department  cost  per  graduate  for  each  department. 

(e)  Add  cost  per  graduate  for  each  department  to  arrive  at  cost  per 
graduate  per  course. 

b.  Computation  of  indirect  costs. 

(1)  Indirect  costs  consist  of  a  pro  rata  share  of  base  operations,  medical, 
training  aids,  family  housing  admin,  and  TOE  support  of  training. 

(2)  To  compute  costs  per  graduate  the  following  steps  will  be  taken: 

(a)  Determine  school/AlC  share  of  base  overhead  in  the  following  manner: 

B0000 ,  G0000 ,  H0000 ,  JOOOO  and  N0000  -  average  population  of  school/ATC 
(including  students,  trainees,  overhead,  staff  and  faculty,  etc.)  -  total  post 
population  (assigned,  attacned,  tenant  units,  TOE  units,  trainees , *s tudents , 
etc.  minus  those  assigned  to  base  operations  accounts)  =  school/ATC  percentage 
share  of  these  accounts.  NOTE:  Rationale  for  this  computation  is  that  base 
overhead  personnel  and  services  are  provided  for  tenants,  schools,  TOE  units, 
etc.,  and  that  if  these  activities  were  not  present,  there  would  be  no  need 
for  the  base  cveilnvul.  Therefore,  huso  overhead  is  distributed  t.c  .ill 
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activities  on  the  installation  other  than  the  base  overhead  itself.  Part  of 
base  administration  is  not  distributed  to  maintenance  and  vice  versa. 

C0000  -  End  items  processed  for  school/ATC  i  (divided  by  total  end  items 
processed  -  those  processed  for  base  operation;:  accts)  =  school/ATC 
percentage  share  of  COOOO. 

K0000,  L0000  and  MOOOO  -  square  footage  of  bldgs  and  facilities  assigned  to 
school/ATC  -  (total  installation  square  footage  -  square  footage  assigned  to 
base  operations  accts)  =  school/ATC  percentage  share  of  K0000,  L0000  and  MOOOO. 

1900,720000,  840000,  390000,  and  818097.2  use  population  calculations  as  indicated 
for  B0000,  G0000,  etc.,  except  in  840000  account  include  retirees  and  dependents 
in  determining  total  installation  population. 

(b'  Multiply  percentages  derived  in  paragraph  1  above  times  total  instal¬ 
lation  civilian  pay,  supplies  and  equipment,  other  OMA,  and  MPA  costs  in  the 

appropriate  base  operations  accounts  to  arrive  at  school  dollar  share  in  each 
account, 

(c)  Divide  school  share  of  each  account  by  total  school/ATC  manweeks  of 
training  to  arrive  at  cost  per  manweek  for  each  account. 

(d)  Multiply  the  cost  per  manweek  for  each  account  by  the  manweeks  per 
graduate  for  each  course  to  arrive  at  the  cost  per  graduate. 

(c)  Add  MPA  costs  and  OMA  costs  for  E0000  through  N'OOOO  to  determine  base 
operations  cost  per  graduate  in  MPA,  civilian  pay,  supplies  and  equipment,  and 
other. 

(f)  Installation  mission  costs  are  displayed  separately  in  OMA  and  MPA 
for  each  account. 

(3)  TOE  Support  Costs. 

(a)  Personnel. 

1  Compute  mandays  by  grade  devoted  to  each  course. 

2  Multiply  by  number  of  classes. 

—  Multiply  result  obtained  in  2  above  by  daily  pay  rate  to  arrive  at  cost 
by  grade. 

—  ^um  costs  for  all  grades  of  TOE  personnel  supporting  courses. 

5  Divide  result  by  number  of  normalized  graduates  to  arrive  at  TOE,  MPA 
support  cost  per  graduate. 
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6  Repeat  for  each  course  that  receives  TOE  support. 

(b)  Equipment. 

JL  Determine  equipment  days  by  type  utilized  by  TOE  units  in  support  of 
each  course. 

2  Multiply  by  number  of  classes. 

Compute  daily  depreciation  costs  for  each  piece  of  equipment.  (Acquisition 
cost  *  amortization  period  in  years  *  365  =  daily  depreciation.) 

4  Multiply  results  obtained  in  2  by  ^  to  obtain  total  depreciation  cost  by 
equipment  type. 

Jj  Sum  costs  for  all  equipments  used  in  support  of  course. 

6  Divide  by  number  of  normalized  graduates  to  arrive  at  PEMA  equipment  cost 
per  student. 

2  Repeat  lor  each  course. 

c.  Total  and  variable  direct  (mission)  and  indirect  (base  operations). 

(1)  Mission  costs. 

(a)  Using  student  manycars  (average  yearly  student  load)  as  the  independent 
variable  and  OMA  mission  costs  for  each  school  and  training  center  as  the 
dependent  variable,  perform  a  2  variable  linear  regression  with  5  years  of  data 
available  in  the  CON'ARC  data  bank.  (NOTE:  Hiis  has  already  been  accomplished 
for  OMA,  utilizing  4  years  of  data  in  the  development  of  budget  cost  factors.) 
Determine  the  fixed  OMA  cost  and  the  variable  cost  per  s tudent/ trainee  year. 

(b)  The  slope  of  the  regression  line  (b  in  the  equation  Y  **  a  +  bx)  will 
represent  the  variable  cost  per  student  year. 

(c)  Using  the  same  technique  as  described  in  (a)  and  (b)  above,  determine 
the  fixed  cost  and  variable  MPA  cost  per  student  manyear.  Total  MPA  for  the 
school  or  training  center  will  be  used  as  the  dependent  variable.  NOTE:  Student 
pay  and  allowances  and  per  diem,  when  authorized,  will  be  added  to  the  variable 
MPA  cost  per  student/trainee. 

(d)  Using  the  fixed  OMA  and  MPA  costs  developed  for  each  school  and  ATC 
by  the  2  variable  regression  directed  in  c,  (1)  (a)  and  (c)  above,  compute  the 
variable  cost  per  student  per  course  in  the  following  manner: 

Total  fixed  cost  for  school  or  ATC  =  $2,500,000 

Total  student  manycars  for  entire  school  or  ATC  =  3,000 

Student  manycars  for  course  X  =  120 

Percent  of  fixed  cost  attributable  to  course  X  =120  =  .4  percent 

3000 
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Fixed  Cost  for  Course  X  =  $  10,000 

Total  cost  for  Course  X  (determined  in  para  5b)  =  $120,000 
Variable  cost  for  Course  X  =  $120,000  -  $10,000  =  $110,000 
Normalized  graduates  Course  X  =  250 

Variable  cost  per  graduate  for  Course  X  =  $110,000  =  $440 

250 

(e)  The  accuracy  and  validity  of  the  fixed  and  variable  breakout  will  be 
tested  in  the  following  manner: 

Fixed  cost  per  student  per  course  X  number  of  normalized  graduates  =  fixed  cost 
per  course. 

Summation  of  fixed  cost  per  course  for  all  courses  =  total  fixed  cost  for 
school/ATC. 


Variable  cost  per  student  per  course  X  number  of  normalized  graduates  =  variable 
cost  per  course. 

Summation  of  variable  costs  per  course  =  total  variable  cost  for  school/ATC. 

Total  fixed  cost  for  school  or  ATC  +  total  variable  cost  for  school  or  ATC  = 
total  computed  cost. 

Total  computed  cost  =  total  reported  cost  for  school  or  ATC. 

(f)  PEMA  costs  per  student  will  not  be  broken  out  into  fixed  and  variable 
segments.  PEMA  costs  attributable  to  a  student/traincc  are  those  generated  by 
expenditure  of  ammunition  and  depreciation  of  major  items  of  equipment  used  in 
training.  Ammunition  is  normally  used  by  the  student/trainee  and  no  portions  of  it 
can  reasonably  be  allocated  to  the  "fixed"  element  of  a  school,  training  center 
or  department.  Therefore,  it  will  all  be  considered  as  a  total  cost  directly 
chargeable  to  the  student/trainee  and  the  amount  expended  per  course  or  ATP  will 
depend  on  the  student/ trainee  input.  PEMA  equipment  issued  to  a  school  or 
training  center  depreciates  at  a  rate  that  is  dependent  upon  its  expected  usable 
life  and  its  intensity  of  use,  however,  many  items  of  equipment  become  obsolete 
prior  to  "wear  out"  from  use  by  students  and  trainees.  Under  these  conditions, 
that  is  when  the  equipment  is  depreciating  at  seme  unknown  rate  merely  because 
it  has  been  issued  to  a  school  or  training  center  regardless  of  its  use  by 
s tudcnts/trainecs ,  there  is  no  knovm  accurate  method  for  determining  that  portion 
of  depreciation  costs  which  are  fixed  and  that  portion  which  vary  with  student/ 
trainee  load  and  hence,  intensity  of  use.  Therefore,  costs  related  to  deprecia¬ 
tion  of  equipment  used  in  training  will  be  computed  in  total  only. 

(2)  Base  operations  costs. 

('  )  Fixed  and  variable  base  operations  OMA  cost  factors  developed  during 
the  Maroun  Study,  and  which  are  to  be  refined  and  updated  in  accordance  with 
DA  letter,  ACAA  COMPT-SP,  2  Apr  71,  subject:  Analysis  of  CONARC  Base  Operations 
Funding  (Maroun  Study)  will  be  used  as  a  base  point  for  determining  the  variable 
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cours^or^ATp50  J“ribuL:lb1'  '“ch  student/trainee  1„  each 

r  ATP.  The  following  specific  methodology  will  be  employed: 


Total  base  operations  OMA  cost  per  praduate  nor  cnnr«./ATP  _  j  j  .  , 

Analysis  Studies  of  Schools  and^raining  Jenters  (par^lb)."  "  ^ 


baSe  »P«™tioM  OMA  coat  per  graduate  per  course  -  variable  cost  Is 
Whici  schoTo?  A°K  Ji  IS"  0P"ati0"S  0f  »P« 


MWng'—  StUdCnt  P"  “““  W1U  bC  ‘“"P““d  bS  *»«««  in  the 


Variable  OMA  Base  Opns  Cost  per  MMY  = 
Course  Length  for  Course  X  = 
Military  Manyear  = 
Variable  Cost  per  Student  for  Course  X  - 


$400 
20  weeks 
52  weeks 
20  x  $400 
52 


“  $153.85 


(c)  Upon  completion  of  the  current  Prnornm  cm  *.  »  , 

Will  be  developed  which  will  p  rovide  for  the  Tv  oca  inn  I  f  Si  ?  mc,thodo,°^ 
medical  costs  to  each  s tuden t/tral nee  f raduatod ‘ f ron  CONARC  sth'  VariabU' 
centers.  The  methodology  developed  „m  be  appended  to' “is  plan!'  ^ 


costs1^  sy;„  ii\izTx\ortntion- uxcd 

for  each  course  or  AT?  as  follows:  H.ed  O'U  co  var  ble  oma”  "r”'’ 
MPA  cost,  variable  MPA  cost  and  PD1A  cost.  mailable  cost,  fixed 


5.  Data  Adjustments. 
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b.  All  dollars  used  in  this  analysis  have  been  inflated  to  the  FY  72  level. 

6.  Attached  at  Inclosurc  1  are  costs  per  graduate/trainee  for  each  course 
taught  during  FY  71,  broken  out  by  direct  and  indirect  in  the  three  appropri¬ 
ations  of  OMA,  MPA  and  PEMA,  and  further  broken  out  by  total  fixed  and  variable 
in  OMA  and  PEMA. 
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DEPARTMENT  OF  THE  AIR  FORCE 

AFHRL  TECHNICAL  TRAINING  DIVISION  (AFSC) 
LOWRY  AIR  FORCE  BASE,  COLORADO  60230 


20  October  1972 


Thomas  Marshall 
Stanford  Research  Institute 
Menlo  Park,  CA  94025 

Dear  Tom: 

As  indicated  in  our  telecon,  I  am  sending  along  some  material  to  assist 
you  in  your  effort  to  document  cost  savings  made  through  the  application 
of  instructional  technology. 

Four  separate  studies  are  covered  in  this  letter,  two  of  which  are  in 
the  final  stages  of  completion  just  before  publication,  while  the  other 
two  are  already  documented  as  Technical  Reports.  Let  me  begin  by 
describing  the  former  two  in  some  detail. 

These  two  contracts  have  been  completed  in  the  Precision  Measurement 
Equipment  (PME)  course  and  the  Inventory  Management  (IM)  course  here 
at  Lowry.  Each  contract  dealt  with  only  selected  units  of  instruction 
in  each  course.  The  first  completed  contract  resulted  in  the  individualiza¬ 
tion  of  one  block  of  instruction  in  the  PME  course.  The  work  was  done 
by  Logos,  Inc.  The  second  contract  resulted  in  the  individualization  of 
selected  lessons  in  Blocks  I  and  III  and  all  of  Block  IV  in  the  IM  course. 
This  work  was  done  by  the  System  Development  Corporation. 

The  block  of  instruction  individualized  in  the  PME  course  covered  two 
major  skills-precision  soldering  and  troubleshooting  electronic  test 
devices.  The  main  purpose  of  the  contracted  effort  was  to  determine 
feasibility  of  individualizing  the  particular  class  of  materials  found 
in  the  selected  block  of  instruction  and  not  with  trying  to  compare 
group  with  individualized  instruction.  Logos  developed  a  highly 
mediated  set  of  lesson  materials  to  teach  the  task  of  precision  solder¬ 
ing.  Their  approach  involved  the  use  of  tape  cassettes,  8mm  film  loops, 
and  picture  books  in  addition  to  the  normal  technical  equipment  needed 
to  perform  this  type  of  task.  The  instructional  strategy  for  teaching 
precision  soldering  centered  on  the  use  of  an  imitative  mastery  model. 
Implementation  of  the  developed  materials  showed  that  all  students  were 
able  to  achieve  the  stated  objectives  with  an  average  40%  savings  in 
time.  Since  criterion  testing  was  utilized,  no  direct  performance 
comparison  can  be  made  to  the  group  instruction.  With  criterion  testing, 
a  student  was  not  allowed  to  proceed  until  he  achieved  a  stated  objective; 
thus,  for  all  students,  a  100%  level  of  achievement  was  recorded. 

An  important  measure  under  criterion  testing  is  the  number  of  times  it 
takes  a  student  to  reach  the  objective(s) .  For  the  precision  soldering 
lessons,  the  minimum  total  trials  to  criterion  is  14.  The  average 
total  trials  to  criterion  attained  was  18.6.  A  further  breakdown  of 
this  information  is  provided  in  attachment  1. 
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Logos  used  an  instructional  strategy  based  on  the  use  of  algorithms  to 
teach  troubleshooting  of  precision  test  equipment.  This  methodology 
resulted  in  a  38"  savings  in  instructional  time  witn  all  students  achiev¬ 
ing  the  stated  criterion  objectives.  The  algorithmic  method  relied 
heavily  on  the  use  of  printed  materials  and  required  frequent  involve¬ 
ment  of  instructors  to  set  up  training  equipment  for  the  student's  use. 
Instead  of  the  instructor  setting  up  equipment  once  for  a  group  of 
students,  the  individualization  of  instruction  required  an  instructor 
to  inject  a  malfunction  in  a  piece  of  test  equipment  for  each  student. 

The  management  burden  resulting  from  this  situation  tended  to  negate 
the  tfficiency  gains  on  the  part  of  the  students.  Attachment  2  gives 
the  times  to  mastery  for  troubleshooting. 

In  the  Inventory  Management  (IM)  course,  the  last  block  of  instruction 
(Block  IV)  and  a  total  of  five  lessons  in  Blocks  I  and  III  were  in¬ 
dividualized.  Except  for  one  lesson  that  dealt  with  the  use  of  a  tele¬ 
type  terminal,  the  material  taught  in  the  selected  IM  lessons  was 
conceptual  in  nature  and  involved  procedural  learning  tasks.  The  main 
body  of  all  lessons  was  taught  through  the  use  of  programmed  texts 
which  allowed  branching  within  themselves.  Selected  lessons  also  had 
a  sound/slide  presentation  which  served  as  both  an  advance  organizer 
and  as  a  review  item.  The  results  of  the  SDC  contract  are  summarized 
in  attachment  3-6. 

Attachments  3  and  4  give  the  results  for  the  small  group  validation 
of  the  IM  instructional  materials.  Attachments  5  and  6  give  the 
results  for  the  large  group  operational  tryout  of  the  materials  and, 
in  addition,  allow  comparison  of  the  small  group  results  with  the 
large  group  results.  Large  group,  operational  tryout  of  instructional 
materials  only  occurred  for  Block  IV.  The  results  of  the  operational 
tryout  showed  a  47%  savings  in  time  with  a  91%  average  achievement  score. 
The  conclusion  that  can  be  drawn  from  the  work  done  in  the  PME  and  IM 
courses  is  that  individualization  of  instruction  can  reduce  or  eliminate 
inefficiencies  that  are  inherent  in  group  instruction  resulting  in 
considerable  reduction  of  training  time  without  any  loss  in  achievement. 

The  second  pair  of  studies  will  be  discussed  only  briefly.  Abstracts 
and  summaries  of  these  studies  are  included  as  attachments  7  and  8. 
Attachment  7  deals  with  the  training  of  Medical  helpers  at  Sheppard 
AFB.  Note  that  only  a  very  small  portion  of  the  course  was  altered 
via  the  -introduction  of  audio-visual  techniques;  namely,  a  single 
chapter  from  a  programmed  text.  The  major  benefit  was  time  saved 
through  elimination  of  redundancy.  With  respect  to  the  two  training 
methods,  there  were  nonsignificant  differences  in  performance  on  a 
test  covering  the  material  taught.  The  75%  time  savings  claimed  here 
can  be  generalized  only  with  caution. 
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Attachment  8  is  included  to  point  up  a  problem  of  which  I  am  sure  you 
are  aware.  Here  there  was  no  time  saved  (in  fact,  on  the  average, 
tne  amount  of  time  spent  reading  the  material  doubled),  but  statis¬ 
tically  significant  gains  in  performance  were  registered.  How  to 
evaluate  gains  in  performance  using  a  dollar  criterion  remains  a 
major  obstacle  to  assessing  the  overall  cost  effectiveness  of  intro¬ 
ducing  instructional  innovation. 


Incidentally,  during  my  visit  to  Randolph  earlier  this  week,  I  alerted 
Chet  Bueker  and  Don  Meyer  that  you  might  be  contacting  them.  If  you 
have  any  questions  regarding  the  attached  material,  or  we  can  be  of 
assistance  in  any  other  way,  feel  free  to  give  me  a  call. 


Most  cordially, 


^  f  /TSi  -st 

MARTY  R.^ROCKWAY 
Technical  Director 


8  Atch 
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Table  III-ll.  Stage  2  Validation  Student  Learning  Time  Data  Summary 
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Table  HI-15.  Small/Larce  Group  Performance  Comparison 
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of 

Item 

Moan 
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Stage  3 

Students 
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Score 

Students 

Number 

Score 

I 

1 

4 

21 

77 

20 

21 

93 

+ 

16 

4 

61 

72 

20 

60 

89 
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17% 
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85 

20 

53 

92 

+ 

7  y 
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41 

88 

20 
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93 
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89 

20 
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20 
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91 

85 

+ 
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152  i 

4 
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87 

20 
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+ 
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81 

91 

+ 
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Table  III-16  presents  summary  learning  time  data  for  the  eight  Block  IV  lessons. 
Raw  learning  time  data  for  each  student  are  located  in  Appendix  C  of  this  report. 

Table  III-16,  Stage  3  Validation  Student  Learning  Time 

Data  Summary 


Block  IV 
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Hours 

Mean 
’/  Saved 

1 

1:23 

3 

54 

2 

7:06 

9 

21 

3 

4:24 

9 

53 

4 

3:01 

6 

50 

5 

2:40 

6 

56 

6 

3:13 

6 

46 

7 

1:38 

3 

4c 

8 

0:33 

3 

82 

23:58 

45 

47 

,4  7  77*  A  r 


128 


Table  III-17  compares  time  savings  oc 
those  in  the  Stage  3  validation. 


ins  ir.  the  Stage 


validation  vit'.' 


As  shown  in  the  table,  time  savings  in  the  large  group  trials  were  not  nmarty 

so  dramatic  as  those, in  tne  small  group  t*...a.<.s,  al.I.c-j..  t..e 

large  group  trials  more  than  meet  the  criterion  estaolishod,  wmch  wan  oo  ,. 

The  Stage  3  time  savings  are  more  realistic  predictors  or  lesson  it-  -  • 

course  as  it  will  be  given,  since  conditions  in  the  Stage  3  trials  mere  n—r  . 
approximate  actual  course  conditions. 


Student  attitude  data  are  summarized  in  Table  HI-13 
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SUMMARY 


Federico,  P.A.  Evaluating  an  experimental  audio-visual  module  programmed  to  teach  a  basic  anatomical  and 

physiological  system.  AF11RL-TR-7I-37.  Lowry  AFB  Colorado:  Technical  Training  Division,  Air 

Force  Human  Resources  Laboratory,  July  1971. 

Problem 

Air  Force  enlisted  medical  and  dental  personnel  begin  their  training  with  the  Medical  Service 
Fundamentals  Course  at  the  Medical  Service  School  (MSS),  Sheppard  Air  Force  Base,  Texas.  Included  in 
the  core  curriculum  arc  units  of  instruction  planned  to  teach  rudiments  of  medical  terminology,  anatomy, 
and  physiology.  Currently,  MSS  is  using  a  linear  programmed  text  to  teach  these  subjects  which  are  needed 
as  a  basis  for  understanding  structures  and  functions  of  major  mechanical  and  chemical  systems  of  the 
human  body.  Despite  precautions  taken  by  the  author  of  this  text,  training  managers  at  the  MSS  were 
concerned  that  some  of  their  less  literate  trainees  were  having  difficulty  following  the  programmed  text.  It 
was  thought  that  students  were  not  given  sufficient  exposure  to  audio-visuals  to  enable  them  to  view 
subject  matter  in  an  integrated  way,  or  to  understand  its  overall  organization.  The  primary  purpose  of  this 
study  was  to  detemiine  whether  a  newly  developed  audio-visual  programmed  module  was  more  effective 
and  efficient  than  the  currently  used  programmed  text  for  teaching  an  instructional  segment  on  the 
digestive  system  in  the  course.  A  secondary  purpose  was  to  estimate  student  opinions  and  reactions  to  this 
audio-visual  programmed  approach  of  teaching  this  system,  as  opposed  to  the  written  programmed 
approach. 

Approach 

An  audio-visual  programmed  module  was  developed  to  teach  the  rudiments  of  the  digestive  system.  It 
consisted  of  synchronized  35mm  color  slides  and  an  audio-visual  tape;  and  it  covered  the  same  enabling 
objectives  as  the  corresponding  chapter  from  a  linear  programmed  text.  However,  the  script  was  not  as 
redundant  as  the  written  programmed  material.  Air  Force  enlisted  trainees  had  been  randomly  and 
previously  placed  into  four  classes;  and  educational  media  were  randomly  assigned  to  these  classes 
according  to  a  Solomon  tour-group  experimental  design.  Statistical  comparisons  of  the  groups  were  made 
in  terms  of  posttest  performance.  Also,  a  theme  analysis  was  conducted  of  student  reactions  and  opinions 
toward  the  audio-visual  program  and  written  program. 

Results 

It  was  demonstrated  that  the  audio-visual  programmed  approach  to  training  was  more  efficient  than 
and  equally  as  effective  as  the  programmed  text  approach.  It  was  possible  to  reduce  the  duration  of  a  lesson 
as  much  as  75  to  83  percent  while  still  maintaining  its  elfectivcncss  by  using  a  minimally  redundant 
audio-visual  instructional  module.  Trainees  of  different  intellectual  abilities  acquired  as  much  knowledge 
about  the  digestive  system  from  viewing  the  20-minutc  audio-visual  module  as  from  interacting  for  80  to 
120  minutes  with  the  programmed  text.  Students  who  differed  in  their  mastery  of  the  rudiments  of 
anatomy,  physiology,  and  medical  terminology  performed  equally  well  after  audio-visual  instruction  or 
after  written  programmed  instruc  ion.  Trainees  reported  more  positive  reac  lion  to  the  audio-visual  program 
than  to  the  written  program. 

Conclusions 

In  view  of  the  findings,  it  was  recommended  that  within  the  Medical  Service  Fundamentals  Course 
the  utilization  of  audio  visual  programs  be  maximized  and  the  utilization  of  written  programs  be 
minimized. 

This  summary  w^s  prepared  by  Pat-Anthony  Federico,  Technical  Training  Division,  Air  Force  Human 
Resources  Laboratory. 
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SUMMARY 


Sellman,  Wayne  S.  Effectiveness  of  experimental  training  materials  for  low '  ability 

AFHRL-TR-70-16.  Lowry  AFB,  Colo.  Technical  Training  Division,  Air  Force  Human  Resource* 

Laboratory,  June  1970. 


Problem 

With  the  influx  of  New  Menial  Standards  airmen,  many  Air  Force  training  experts  have  expressed 
concern  that  existing  career  development  courses  (CDCs)  aie  written  at  a  level  which  is  too  di  icu  t  to  e  a 
satisfactory  training  device  Recognizing  the  need  for  training  materials  appropriate  foruse :  by  low  aptituae 
personnel,  CDC  wnters  at  Chanute  Technical  Training  Center  developed  a  new  CDC  format  expressly 
designed  to  reduce  some  of  the  learning  skill  requirements  imposed  by  conventional  CDCs.  Ihe  present 
study  was  designed  to  determine  if  modifying  CDC  format  through  the  simplification  of  written  materials 
and  addition  of  audio  supplementation  could  improve  the  CDC  as  a  training  device.  The  expenmental  Luu 
was  especially  designed  as  an  aid  to  teach  basic  job  information  to  airmen  possessing  minimal  verbal  skills. 

Approach 

One  hundred  eighty  technical  school  students  at  Lowry  Air  Force  Base,  Colorado,  subdivided  into 
high,  middle,  and  low  aptitude  groupings,  served  as  experimental  subjects.  Mental  category  status  of  each 
airman  was  defined  in  terms  of  his  score  on  the  Armed  Forces  Qualification  Test  (AFQT),  the  measuring 
instrument  used  to  determine  whether  or  not  an  individual  meets  acceptable  mental  siandards  for  induction 
or  enlistment.  Three  versions  of  the  CDC  for  the  57130,  Fire  Protection,  career  ladder  were  used  in  the 
study:  («j)  a  conventional  CDC  as  published  by  the  Extension  Course  Institute  and  used  Air  Force-wide  for 
on-the-job  training,  (b)  an  experimental  CDC  with  lowered  reading  requirements,  additional  illustrations, 
and  a  double -spaced,  single  column  format,  and  (c)  the  same  experimental  CDC  with  an  accompanying  tape 
recording  which  reiterated  material  presented  in  the  CDC.  Criterion  measures  included  a  fire  protection 
achievement  test,  the  length  of  time  required  to  read  the  CDC,  and  a  questionnaire  concerning  subject 
attitude  toward  the  CDC  Twenty  subjects  from  each  mental  ability  category  were  asked  to  study  each  CDC 
and  respond  to  the  test  and  attitude  questionnaire. 

Results 

The  results  indicated  that  modifying  the  conventional  CDC  by  reducing  the  reading  level,  augmenting 
the  number  of  illustrations,  and  changing  the  format  to  a  single  column,  double-spaced  page  produced 
statistically  significant  differences  in  learning  scores.  Middle  and  low  aptitude  personnel  showed  greatest 
learning  using  the  CDC  with  audio  supplementation.  High  ability  personnel  showed  only  slight  differences 
across  all  CDCs.  High  and  middle  aptitude  personnel  consistently  outperformed  the  low  aptitude  group 
across  all  CDCs,  and  they  demonstrated  faster  reading  time?  as  well. 

Conclusions 

CDCs  should  be  written  at  a  reading  level  appropriate  for  the  majority  of  airmen  in  the  career  ladder. 
Additional  illustrations  and  pictorial  materials  also  seem  necessary.  The  comparisons  of  CDCs  indicated  that 
most  personnel  learned  more  when  using  the  CDC  with  audio  supplementation.  However,  more  definitive 
research  and  analyses  must  be  completed  to  determine  if  this  will  be  a  continuing  trend  across  all  aptitude 
ranges  and  career  ladders. 

This  summary  was  prepared  by  Wayne  S.  Sellman,  Technical  Training  Division,  Air  Force  Human 
Resources  Laboratory. 


